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AN EXPERIMENT IN TEACHING LABORATORY 
PSYCHOLOGY BY THE PROJECT METHOD! 


MARY SHIRLEY ann KATE HEVNER 


University of Minnesota 
PURPOSE AND HISTORY OF THE EXPERIMENT 


The purpose of this study is to evaluate a project method of 
teaching the elementary psychology laboratory course by com- 
paring two groups of students taught by this method with two 
other groups of students taught by the same instructors under 
the same conditions but by the ordinary routine method. The 
study was carried on by the writers during the academic year 
1928-29 with laboratory classes in the Department of Psychol- 
ogy at the University of Minnesota, and includes results from 
six class groups comprising 226 students. 

Like the elementary laboratory course in any science, the 
primary aim of the laboratory course in psychology was to 
introduce the student to scientific method in general and to 
give him acquaintance with specific methods and tools of 
psychological science in particular. In the psychology labora- 
tory, therefore, the student was drilled in simple graphic meth- 
ods, and in the statistical devices commonly used by psycholo- 
gists, and was given instruction in the methods and techniques 


1 The writers wish to acknowledge their indebtedness to Prof. Miles 

A. Tinker, director of the laboratory course, for advice during the ex- 
periment and for reading the manuscript. His experience with the proj- 
ect course the previous year made him an able and helpful critic. The 
authors also thank Prof. R. M. Elliott for his encouragement and sup- 
port of project work and for his careful criticism of the manuscript. 
Finally to Prof. D. G. Paterson for his enthusiastic approval when the 
plans for introducing project work into the laboratory course were first 
laid before him and for his suggestion of the experimental study Mary 
Shirley expresses her gratitude. 
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of psychological experimentation by repeating a number of the 
standard experiments. The object of this study, then, in more 
definite terms was to discover whether the routine method or 
the project method was the more effective means of teaching 
scientific method and statistical analysis and of arousing the 
students’ interest in psychology. 


METHOD OF THE PRESENT EXPERIMENT 


Preliminary experimentation with the project method was 
carried on in the department at Minnesota for two years pre- 
vious to 1928, and data were collected from five different proj- 
ect classes and their contro] classes. This earlier experimenta- 
tion has been regarded merely as orientation work; a short 
summary of this exploratory work will make clear the status 
of the project experiment at the beginning of this study. 

The first trial of this method was made in the spring term, 
1927, by an instructor who after eight terms experience as 
assistant and instructor in the routine laboratory course felt 
the necessity of making the course more interesting from the 
teacher’s point of view. Incidentally it was hoped that by 
linking routine experiments with special projects the students 
would become interested in the results of experimentation and 
would therefore do more careful experimenting. It was planned 
to use data from certain routine experiments such as maze 
learning, mirror drawing, Army Alpha, etc., for correlation with 
data from tests of radicalism (after the manner of Moore), 
emotionality, and the like, which were to be worked up as 
special projects. So the students carried out practically all 
the routine experiments with the knowledge that they were 
collecting material and learning methods to be used later in 
their own projects. At the end of the term the class as a whole 
had completed 13 projects in addition to the routine work. 
These projects were reported by the students responsible for 
them before the class at its final meeting. Four of the reports, 
which were adaptations of experiments done by well known 
psychologists, were sent to the original investigators and the 
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students subsequently received favorable comments on their 
work. Two of these projects have recently been published.? 

No attempt at controlled experimentation was made in this 
first preliminary trial of the project method and the students 
were assured repeatedly that the instructor was not exploiting 
them for her own ends. But sinee the students of this class 
seemed more interested in the course than did students in the 
regular sections their attitude and criticisms of the course were 
obtained by a rating scale questionnaire. The questionnaire 
was given to the routine class which met during the same term 
and the judgments of the two classes were compared to deter- 
mine whether it was worth the Department’s while to sponsor 
a better controlled investigation of the project method. 

The following year two instructors each taught one regular 
and one project section in the winter term and one section in 
the spring term in which the students were allowed to choose 
either project or regular work. Differences between project and 
regular students were studied by the results of new type “‘objec- 
tive” examinations and questionnaires. From this evidence it 
was discovered that the students in the regular sections did 
slightly better on the examination, but that students in the 
project sections were more interested and felt better profited 
by the course. It seemed worth while to continue the project 
course in an experimental way. 


NATURE OF THE LABORATORY COURSE AT MINNESOTA 


The present experiment (1928-29) has been carried out under 
as rigidly controlled conditions as the circumstances of an elec- 
tive course in which different students meet at different class 
hours under different instructors permit. At the University of 
Minnesota five sections of psychology laboratory are taught 
during the fall and winter terms and two during the spring. 


2? Anderson and Dvorak: Difference between college students and 
their elders in standards of conduct. Jour. Abn. and Soc. Psych., 1928, 
23, 286-292. 

Wheeler and Jordan: Change of individual opinion to accord with 
group opinion. Jour. Abn. and Soc. Psych., 1929, 24. 
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Six of these seven classes were taught by the writers and are 
therefore included in the experiment. There is no difference 
in the content of the course for these sections. However, in the 
fall and winter terms each class meets for a two-hour period 
two days a week for the two terms, while in the spring each 
class meets for a two-hour period four days a week. In either 
case the entire course carries four hours of credit. 


PERSONNEL: INSTRUCTORS AND STUDENTS 


The instructors, both women, were approximately equal in 
training and in previous teaching experience. During the fall 
and winter each instructor taught two sections; during the 
spring each taught one. Two graduate assistants were assigned 
to each section. The instructor gave all the general explana- 
tions, demonstrations, and lectures on statistical methods. 
The graduate assistants had charge of apparatus, went about 
the laboratory from booth to booth to see that each student 
set up the experiment correctly and properly understood the 
method, assisted the instructor in giving individual aid, and read 
the students’ notebooks. 

Most of the students in the course were sophomores in the 
College of Science, Literature and the Arts; some were juniors 
and seniors, and a few were registered in the College of Educa- 
tion. Several students were enrolled in the school of Music, 
and some were pre-legal students, since the laboratory course 
is particularly recommended to such students. Most of the 
students took the laboratory course parallel to the general 
course in psychology or else had taken the general course in the 
previous term. The laboratory course is, of course, not open 
to students who are not taking or have not taken the introduc- 
tory course in general psychology. Many students register 
for the laboratory course in order to fulfill the requirement of 
one laboratory science for entrance to the senior college. About 
two-thirds of the students were women. Each section was 
limited to 48 students but none of the sections were filled to 
capacity. 
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TEXT AND MATERIAL PRESENTED 


The text used in the laboratory course was Foster’s Experi- 
ments in Psychology. Since the text was originally written for 
the Minnesota course the experiments are particularly well 
adapted to local conditions. The experiments described in the 
text were supplemented by a few other experiments and demon- 
strations at the instructors’ discretion. Some attempt was 
made to make the laboratory experiments parallel the topics 
under discussion in the general course. The knee-jerk was 
studied as an example of a typical reflex at the time the ner- 
vous system was under discussion in the general course; experi- 
ments on human and animal learning and on mental set came 
while those topics were being taken up in the introductory 
lectures, and Army Alpha and the Seashore test of musical 
ability, and the Kent-Rosanoff association test were given 
toward the end of the course after the lecturers had discussed 
intelligence tests, special aptitude tests, and free association. 
However no effort was made to link the two courses completely 
or to hold the laboratory students responsible for material 
presented in the general course. 

During the fall term the instructors were in some respects 
hindered and in others helped by the fact that the Foster manual 
was undergoing revision. New experiments in process of try- 
out and prepared in brief mimeographed form required the 
instructor to carry more of the burden of discussion and ex- 
planation of the problem. The collecting and compiling of data 
from the experiments took extra time both from the students 
and from the assistants. Finally the necessity for trying out 
all the new experiments in all sections imposed a rigid schedule 
on the instructors and made it impossible for them to lead up 
to the project work gradually by having the students assume a 
larger amount of responsibility for the method and results with 
each new experiment. These handicaps were offset by the fact 
that the rigid schedule which was maintained during the first 
term made the teaching methods of the two instructors more 
uniform, and allowed for no divergence whatever in the content 
of the course from section to section. Moreover it is possible 
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that the students’ knowledge that their results would be pub- 
lished as standards for future classes gave them more interest 
in their work and incited them to greater care in performing the 
experiments. At least it seems likely that the reporting of 
individual data and the compiling of class results kept them 
aware of the necessity for conducting psychological experiments 
on a large scale. 

In its general plan the first half of the laboratory course is 
about equally divided between the teaching of statistical and 
graphic methods and the performing of experiments. The 
students are taught to represent distributions by various types 
of frequency graphs, to plot learning curves, and to plot rank- 
order correlations. They are taught to derive measures of 
central tendency, measures of variability, to discover the 
statistical significance of a difference between two averages 
and to compute a rank-order correlation. Nine experiments 
are done during the first half of the course to teach and illustrate 
the use of these methods. 

During the course of this study all students, whether they 
subsequently pursued the regular work or changed to projects 
during the last half of the course, did the same work during the 
first half. Since all the statistical methods were taught in the 
first part of the course more variation was possible in the second 
part. During the second part of the regular course most of the 
experiments were of the nature of psychological tests and meas- 
urements and most of them involved the application of the 
formula for significance of a difference between averages and 
the method of correlation. Eleven experiments were done dur- 
ing this term. In the project section the second half of the 
course was introduced with the Army Alpha test. The re- 
mainder of the term was spent on project work. 


OUTLINE OF EXPERIMENTS CARRIED ON IN REGULAR COURSE 


In the regular course the students work in pairs. For some 
experiments the data are collected by a single group test in 
which the instructor acts as experimenter. As a rule all stu- 
dents are required to write and hand in a report on group experi- 
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ments. Group experiments are used chiefly for teaching new 
statistical methods and consequently it is necessary that each 
student work up the data. About half the experiments are 
done individually, each student of a pair now experimenting 
on his partner, and now acting as subject. In the individual 
experiments only the experimenter is required to write up the 
report. The report is handed in for grading not later than one 
week after the experiment is completed. 

A complete outline of the regular course follows. In this 
outline the word “‘group”’ means that every member of the class 
wrote up the experiment; ‘individual’? means that only half 
of the members of the class wrote up the experiment. 


Outline of Elementary Psychology Laboratory 





EXPERIMENT | METHOD TAUGHT PRINCIPLE PRESENTED 








. Printing H’s, Oppo-| (a) Frequency graphs,| Individual differences 
site Test, Digit; (6) measures of in simple traits rep- 
Cancellation, central tendency resenting distribu- 
Free Association and variability tion by numerical 
(group tests) scores 

2. Knee jerk (individ- | Serial variation, over-| Inhibition and facilita- 
ual experiment) lapping | tion of nerve im- 
pulses, ‘‘all or none’”’ 
law 
3. Set and complex | Differencesandover- | Facilitation of  re- 
behavior (indi- lapping sponse by set 
vidual) 

. Visual Acuity, | Snellen test, Ishihara | Method of testing sen- 
Colorblindness, test, mapping color sory acuity and color 
Color Zones (in- zones vision 
dividual) 

. Blind Spot, Retinal | Perimeter, stereo- | Demonstration of 
Rivalry, Con- scope, and other | special visual phe- 
trast, Color Mix- apparatus | nomena 
ing (individual) 

. Digit Symbol Cod- | Learning curves for | Nature of learning 
ing (group) individuals and process; plateaus, 

group end spurts, curve 

smoothing 

. Maze learning (in- | Learning curve Human trial and error 
dividual) learning 
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Outline of Elementary Psychology Laboratory—C ontinued 

















EXPERIMENT METHOD TAUGHT PRINCIPLE PRESENTED 
8. Memory MethodsI | Paired associates, | Economy of sense ma- 
(individual) prompting, com- terial, value of in- 
plete mastery tent to learn 
9. Animal learning | Curve smoothing Controls used in ani- 
(group) mal experimenta- 
tion; contrast of 
human and animal 
learning 
10. Rational learning | Correlation reli- | Reliability of a test, 
(group) ability validity rational compared to 
trial and error learn- 
ing 
11. Motor learning (in- | Learning at various | Trial and error, inter- 
dividual) stages ference in habit for- 
mation 
12. Span of Visual Ap- | Statistical _signifi- | Greater span for sense 
prehension cance of a differ- material 
(group) ence 
Mid Course Examination 
13. Army Alpha* | Reliability, validity | Methods and princi- 
(group) and sex differences ples of intelligence 
testing 
14. Reading ability | Correlation, differ- | Special aptitude tests, 
(individual) ences eye movements 
15. Seashore Musical | Rating Scale, Per- | Self ratings compared 
Test (individual) centiles, Profile to objective tests 
graph 
16. Attention value of | Orderofmeritmethod| Use of native and 
advertisements acquired attention 
(group) values in advertising 
17. Affective value of | Paired comparison Space errors, factors 
colors (individ- influencing affective 
ual) status 
18. Affective value of | Paired comparison, | Consonance, overtone, 


musical intervals 
(individual) 





absolute ratings on 
a graphic rating 
scale 





and beats, compari- 
son of methods 





* This experiment was the first one done in the second part of the 


course and was the last routine experiment done by project students. 
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Outline of Elementary Psychology Laboratory—C oncluded 














EXPERIMENT METHOD TAUGHT PRINCIPLE TAUGHT 
19. Free Association | Associationfrequency} Community of  re- 
(individual) tables sponse in normal 
subjects 
20. Diagnostic Associ- | Diagnostic symptoms} Advantages and limi- 
ation (group) in free association tations of association 
method in detecting 
} guilt 





Final Winter and Spring Quarter Examination 





CHOICE OF PROJECT SECTIONS 


At the beginning of the winter term each instructor arbi- 
trarily selected one section to do project work and the other to 
continue the regular work. In making this decision the in- 
structor was influenced by the convenience of the hour of 
meeting, and especially by the size of the various sections. It 
seemed important to choose the larger sections, in order to 
facilitate the collecting of data for the individual experiments 
and tests. After midterm exams in the spring each instructor 
announced to her class that both regular and project work would 
be carried on during the remainder of the term. Students who 
wished to undertake projects were allowed to elect project 
work and students who preferred the regular work, to continue 
with it. 

Since the method of teaching a project section differs con- 
siderably from that in the regular section the various steps in 
the procedure are outlined below and are briefly described in 
the paragraphs following. 


Teaching procedure in proiect sections 


1. Announce project course and stimulate interest in it by a brief 
description of previous projects and a few suggestions for new 
ones. Urge students to think out their own problems along 
the lines of their chief psychological interests. 

2. Devise, meanwhile, a number of suitable problems for the 

students in lieu of a shortage of ideas from the students them- 

selves. 
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3. Confer with each student group, helping those who suggest a 
problem to formulate it concretely and ‘‘selling’’ a teacher- 
made problem to those who make no suggestions. 

4. Suggest methods for attacking the problems. 

5. Criticize the students’ tests, apparatus, and methods before the 
experiments are administered to the subjects. 

6. Present all group experiments to the class, and arrange times for 

individual testing. 

. Confer with student groups as to graphic and statistical methods 

to use for an adequate treatment of results. 
8. Arrange for students to report problems before the class. 
9. Teach new statistical methods to individual groups. 
10. Read and evaluate project reports. 


NATURE OF PROBLEMS USED FOR PROJECTS 


The wise choice of problems for sophomore students was the 
instructor’s most difficult task; once this was over she felt that 
half the work of the term was done, in point of ingenuity and 
diplomacy, though not in point of hours. Not that the origi- 
nating of 15 or 20 problems was in itself a difficult task. Every 
psychologist has at his finger tips a half-dozen little new prob- 
lems or corollaries to well established problems which are 
neither fundamental enough nor refined enough to justify the 
expenditure of his own time or even that of graduate or ad- 
vanced major students, but which are within the grasp of sopho- 
mores and within their abilities to investigate. And an instruc- 
tor who has once taught a project section has become acquainted 
with the interests of sophomore students as well as with their 
limitations, and can thus select from his file of problems the ones 
most likely to be successful. Furthermore, each project group, 
whether or not required to do so, suggests ramifications of its 
own problem and extensions of the method to other similar 
fields which serve as a source of problems for the next group. 

Nor were the difficulties of working with sophomore students 
unsurmountable. Naturally their immaturity did impose 
several limitations on the problems to be set, limitations which 
are so apparent that they need be mentioned only in passing: 
their scanty background of scientific facts and methods; their 
lack of acquaintance with psychological literature, their diffi- 
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culties in using periodical indices for reference work, and their 
inability to comprehend the fine points of articles written by 
psychologists for psychologists. Even the most serious limita- 
tion imposed by the brevity of the course and the large numbers 
of students was overcome by the expedient of planning most of 
the experiments as group tests and by arranging for a few groups 
to collect their material from outside sources. Thus 20 stu- 
dent groups planned 20 different experiments, devised apparatus, 
collected data, analyzed it, interpreted the results, and reported 
them back to the class all within 20 class meetings. 

But the real difficulty lay in setting the right problem for 
each student group. Only 20 per cent of the groups formulated 
problems of their own; 30 per cent more suggested a field of 
psychological interest which enabled the instructor to set suit- 
able problems for them. Those sophomores who did set their 
own problems did not err on the side of triviality in their psy- 
chological interests. To the contrary, the questions which 
troubled them were the very ones which have baffled the ex- 
perts in psychology for years. To be sure the students couched 
them in somewhat vague terms; they said, “I’d like to find out 
just what religion does for you,” or “But does a grade really 
show how much you get out of a course?” or “I believe that the 
student’s interest in school subjects depends on his associations. 
How can I prove that?” The instructor’s job in such cases 
was to scale down such ambitious programs, isolating a single 
aspect of the problem which was within the scope of the stu- 
dents and the time limits of the course, and at the same time not 
to distort the problem to such an extent that the student failed 
to see that the problem he eventually solved actually made a 
contribution toward what he wanted to know. Hence she 
talked with the students long enough to clarify hazy ideas. If 
no possible method of attacking the problem occurred either to 
the students or the instructor on the spur of the moment she 
promised to think about it and she returned in a few days with 
a suggested method or with a modification of the problem which 
met with the students’ approval. 

Many who did not suggest a problem of their own either 
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named a field of interest or described the class of subjects on 
which they wished to work. Such students usually accepted 
eagerly any problem which allowed them to work along the 
line of their hobby. The student who said “I don’t know any- 
thing about animal experimentation but I want to work with 
rats’ was set a problem on distributed and concentrated 
practice in animal maze learning. Two young women inter- 
ested in nursery school education, one of whom was a kinder- 
garten teacher, were given a project on imagination in young 
children. A freshman student of architecture and another, 
the daughter of an architect, proposed for their problem a 
study of balance and of architectural types. To another 
student who had access to a group of students in a barbering 
college the steadiness test was suggested as a possible device 
for detecting differences between potential barbers and college 
students. Still another student who wished to discover whether 
he could hypnotize the class by a sort of tom-tom experiment 
was reined down to the less picturesque program of testing the 
effect of a rhythmic monotonous task on subsequent perform- 
ance in arithmetic. 

With 50 per cent of the groups thus thinking for themselves 
along psychological lines the instructor turned her attention to 
the other half who showed less initiative. She formulated a 
handful of enticing problems dealing with phases of student 
life, tests of emotionality, aesthetics, and similar topics which 
have a particular appeal for the sophomore. She then 
“‘peddled”’ them out, offering two or three assorted problems to 
each group and allowing them to pick whichever they liked. 
It required from 10 to 15 minutes of “sales-talk’”’ to point out 
the possibility of the various problems to each group and to help 
them make a satisfactory choice. It was important that the 
instructor catch up every straw which pointed the direction of 
the students’ interest. For the success of the project method 
dependslargely upon the students being assigned problems which 
are interesting and important to them. 

Since, as a rule, more than one student worked on each proj- 
ect it was impossible to have each one equally well satisfied. 
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Often old partnerships were dissolved and new ones formed in 
order that students with similar interests might work together. 
There was no stipulation as to the number of students who were 
to work on each project. Occasionally one student wished to 
work on a project all alone. Usually a group of two was satis- 
factory in size, but if the problem was complex and the students 
worked congenially together three or four students were put 
in one group. As many as six students worked on one project 
which easily lent itself to division into three separate aspects, 
so that they needed to collaborate only in discussion, summary, 
and conclusions. 

While the business of assigning problems was going on the 
students were busy writing a report of the Army Alpha experi- 
ment. The problems having been assigned, the instructor 
conferred with each group for 15 minutes or longer to suggest 
the general method of attack for each problem, e.g., multiple 
choice tests, judgments by the order of merit method, the use 
of a metronome for monotonous stimulation, sources of tests 
of aesthetic judgment, etc. Two selected projects in each 
section were handed over to a graduate student assistant who 
acted as the advisor for the project from this point on. 

The students then worked out the details of their experiment, 
devised tests, set up apparatus, made stimulus cards, and wrote 
out the complete method and submitted these to the instructor 
for criticism.. The instructor went over the material with 
considerable care before it was put in final form for experi- 
mental use, penciling in corrections and suggesting changes in 
method. In most cases it was not necessary to do over the 
entire test or to utterly change the method. But even with 
utmost care on the part of the instructor errors now and then 
crept into the method which made the experiment unsatisfac- 
tory and the results untrustworthy. The limited time pre- 
cluded all but a minimum of preliminary experimentation, but 
since the experiments planned were short and exploratory in 
nature, and since their object was the teaching of psychological 
methods rather than the turnifig out of a finished piece of 
research this limitation was not a serious drawback. 


rman saree 
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When the experiment was ready the instructor presented it 
to the class if it were a group test, simply because she could 
administer the test more expeditiously and with less confusion 
than would have been occasioned by allowing the student ex- 
perimenters to do it themselves. The raw data were imme- 
diately handed over to the experimenters. A few students who 
collected all or part of their data from an outside source were 
excused from class each day after they had served as subjects 
in group and individual tests. 

After the data were collected the instructor, for her own 
benefit in future conferences with the students, listed the ques- 
tions and statistical procedures which each group would need 
to use in working up its particular results and in proving its 
points. She then conferred from 15 to 30 minutes with each 
group, drawing from them by the Socratic method a statement of 
the problem and the statistical treatment to be used. To some 
groups, when conference time was lacking, written directions 
for writing up the experiment were given. Often it was neces- 
sary to teach the students new methods and short-cuts to facili- 
tate their handling of the data. 

Finally at the last class meeting the project experiments were 
reported before the class. In the winter term the instructors 
selected a dozen representative experiments and arranged them 
as a seminar program which was presented before the faculty 
and graduate students of the Psychology department. In the 
spring term the reports were given before the class alone. One 
instructor had each student, whether or not he had done a 
project, write out criticisms of one project and suggest at least 
one problem which could be attacked by similar methods, 
devise methods for solving it, and suggest statistical treatment 
for the results. These written criticisms and suggestions were 
handed to the instructor who scored them as objectively as 
possible. The project reports were handed in just before the 
final examination and the instructors themselves read and 
evaluated the work. 

Grading project work presented some difficulties. Since 
each group had undertaken a different project there was no 
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basis for comparison of the work of different individuals or 
groups. It was also evident that the students in a group did not 
share equally the work of the project. Even though the actual 
amount of statistical computation was the same for each stu- 
dent, one or two students often either assumed or usurped the 
responsibility, took the initiative, and did the thinking for the 
group. As a rule, however, the instructor was aware which 
student had done the best work. But even if the amount of 
work could have been equated for each student and for each 
group it would have been absurd to mark the project notebooks 
on the number of correlations correctly done and the number of 
significant differences computed. The emphasis had to be 
placed on the student’s originality in devising the problem, 
independence in working it out, and acumen in drawing con- 
clusions,—and these facts were known to the instructor largely 
through subjective impressions or untabulatable objective clues. 


OUTLINE OF THE FIFTY PROJECTS 


For the convenience of the reader the 50 projects which were 
carried out in connection with this study are classified and 
briefly outlined below. Those projects which were originated 
wholly or partially by the students themselves, are marked with 
astar. In most cases the students actually worked with much 
larger groups than the entries under “number of subjects” 
would indicate, for often experimenters did not have time to 
pursue individuals who missed one of a series of tests, and the 
eliminations due to these incomplete data were unusually 
large. 
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RESULTS 


Originally 131 students were enrolled in the fall and winter 
classes in laboratory psychology and 94 in the spring classes. 
It was not possible to obtain the rating on the Minnesota College 
Ability examination for some of these students, and therefore 
the objective results are based on the records from 105 students 
in the fall and winter classes and from 72 students in the spring. 


TABLE 1 
Showing the means and probable errors of the means in the Minnesota 
College Ability Examination and in the examination for the first half of 
the course, before the projects were started 











M AND PE FoR | M AND PE FOR | nevyanuwees 2 | 
eactioms smcTioxs ano PE PED 
College Ability Per- 
centiles: 
WOE, isiasns svete 61.0 +2.14/ 70.4 +2.43/9.4 +3.24| 2.9 
(N = 66) (N = 39) 
ae 64.63+2.9 | 66.2 +2.76 | 1.57+4.0 0.4 
(N = 22) (N = 50) 
Laboratory Exami- 
nation: 
| Re pe 69.234 .82 | 65.03+41.17 | 4.20+1.43| 2.94 
S| eee 31.08+ .75 | 30.644 .51| .34+ .9 .38 
(N = 22) (N = 50) 

















Table 1 shows the difference between the two groups in Col- 
lege Ability and in the examination at the end of the first half 
of the course before any of the projects were started. It will 
be noted that in both winter and spring classes the students 
who followed the regular course were slightly superior in College 
Ability as measured by the Minnesota test, and slightly inferior 
in their knowledge of experimental psychology as measured by a 
new-type “objective’’ examination. These differences, how- 
ever, were not large enough to be judged significant. 

Objective measures of the results of the different methods of 
teaching were obtained in the following ways: 
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1. Notebooks. The students were required to write out an 
account of each experiment, regular or project, according to a 
prescribed form. These accounts were read by the graduate 


TABLE 2 


Showing the means and probable errors of the means for two groups in vari- 
ous objective measures 








M AND PEy M anpD PEy DIFFERENCE “ 2 
PROJECT SECTIONS |REGULAR SECTIONS| AND PE: PEp 
Final Examination, 
Part I (infor- 
mation): 
| Re 35.8 + .51/ 34.064 .71) 1.744 .87/| 2.0 
(N = 66) (N = 39) 
| erry -| 35.662 .70) 36.364 .52 70+ .87 8 


(N = 22) (N = 50) 


Final Examination, 


Part II (Meth- 
ods): 
SP ivcdewsnts 25.524 .12| 24.284 .36/ 1.244 .38)| 3.3 
(N = 66) (N = 39) 
ais aaind s 24.734 .48| 24.40+ .38 33+ .6 .54 


(N = 22) (N = 50) 


Final Examination, 


Total: 
RL accsccess 62.264 .71/ 58.03% .90| 4.23+1.1)| 3.9 
(N = 66) (N = 39) 
Ee 61.00+ .97| 61.244 .79 2441.5 18 


(N = 22) (N = 50) 


Final Grade: 





ee 336.74 +5.48 | 290.25+7.78 | 46.49+9.52| 4.88 
(N = 66) (N = 39) 
ES . 320.0+ 8.6| 304.4 +7.2| 15.6 41.12; 1.4 











(N= 22) | (N = 50) | 





student assistants or by the instructors and were graded on a 
five point scale. The students in the regular sections wrote 
fourteen of these notebooks and the students in the project 
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sections wrote eight of the short notebooks and one longer 


report of their individual experiments. 


The grades on these 


notebooks were summed at the end of the course and the longer 











TABLE 3 
Showing the means and probable errors of the means for the two ‘‘matched”’ 
groups 
M anv PEy M anv PEyy DIFFERENCE te 
PROJECT SECTIONS | REGULAR SECTIONS anv PEp PED 
Final Examination 
Part I (Infor- 
mation): 
Winter (N = 
ee 34.60+ .61| 34.062 .71| .542 .93) .5 
Spring (N = | 
22)..........--| 35.662 .70| 36.724 .70| 1.07 + .98} 1.09 
Final examina- 
tion Part II 
(Methods): 
Winter (N = 
39)............| 26.24% .45| 24.284 .36/1.96 + .57| 3.44 
Spring (N = 
22).........---| 24.734 .48) 23.504 .43/1.23 4 .64 1.9 
Final examina- } 
tion, Total: 
Winter (N = 
39)...........-| 60.284 .85} 58.0834 .71)| 2.25 + 1.10) 2.0 
Spring (N = 
_ | ee ee 61.0 + .97| 61.1 +1.0 |10 +1.4 a 
Final Grade: 
Winter (N = 
39)............] 330.0 +5.98 | 290.254 7.78 |39.75 + 9.81) 4.05 
Spring (N = 
| eee 320.0 +8.6 | 307.2 +10.6 {12.8 +13.6 9 




















individual projects were weighted by six. 
was plotted from these sums and each student received a sigma 
score on his notebook work. 


A new distribution 
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2. Midquarter examination. The students were given a 
midquarter examination lasting an hour which contained both 
“objective’”’ and essay-type questions, covering the psychologi- 
cal principles and the statistical methods taught in the course. 
The distribution was plotted for all sections and each student 
received a sigma score. 

3. Final examination. This wasatwo-hour examination which 
was made up entirely of “objective” questions. It consisted of 
two parts, the first part covering the psychological principles 
and the statistical procedures and the second part covering 
methods of procedure for investigating psychological problems 
and methods of analyzing the results. A frequency distribution 
was made for the combined scores for these two parts and the 
sigma standing for each student was calculated. 

In computing the final grade for the course, the sigma scores 
for the notebooks were weighted by three, the midquarter ex- 
amination by two, and the final examination by four. A new 
distribution was plotted from the sums of these weighted scores 
and letter grades assigned on the basis of these computations 
were reported to the registrar’s office. 

Table 2 shows the differences between the two groups on the 
basis of these objective measures. It will be noted that all the 
differences are very small and for the most part are in favor of 
the project students. The largest difference occurs in the final 
grade for the course and this is probably due to the influence of 
the notebook work which counted as a third of the final grade. 
In general the longer notebooks describing the individual ex- 
periments were better done and earned a higher grade than the 
shorter notebooks written by the regular students. 

A further analysis of the results of the two methods of teach- 
ing was made by “matching” the project students and the regu- 
lar students on the basis of their College Ability percentiles and 
their scores on the examination at the end of the first half of the 
course before any of the projects were started. Table 3 shows 
the results for these two groups of students. Each student 
who worked on a project was paired with a regular student 
whose College Ability percentile and score on the mid-course 











TABLE 4 


Relationship between grades in two parts of course 
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| PROJECT REGULAR 
Correlations: | 
Final Examination with College Ability | 
ee ee be a | +26 + .40 
Final Examination, first half with second | 
half of course: 
SIE Sh 6 Oki id eke vss ab gia ob am ee +.71 + .64 
RR SS ae ee ee | +.45 + .56 
Comparison of grades received in two parts of | 
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examination were equal to his. Again the differences are very 
small but are rather consistently in favor of those students who 


worked on projects. 








TABLE 5 
Showing results of the questionnaire on the attitude of the students toward 
the course 
The Difference and Probable Error of the Difference is tabulated for 
all answers made by Winter Quarter students, but the number of 
Spring project students was too few for the probable errors of the per- 
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In an attempt to get the students to evaluate the two types 
of laboratory work at the end of the entire course a question- 
naire was given in which they were asked to compare the first 
half of the course with the second half. In their answers the 
regular students were comparing the routine work of the first 
half with the routine work of the second half; the project stu- 
dents were comparing the routine work of the first half with the 
project work of the second half. To encourage frankness on the 
part of the students the questionnaires were handed in un- 
signed. It is not assumed that the anwers to these question- 
naires represent the students attitudes with complete accuracy 
or adequacy but they do furnish some information of the 
relative points of view of the students in the two groups. 

From a comparison of the winter term sections (table 5) it is 
apparent that the percentage of students who found their work 
more interesting and who felt that they learned more during the 
last half of the course is considerably greater for project stu- 
dents. They also believed that project work required somewhat 
less time than regular work. Project and regular students 
agreed in their opinions of the difficulty and the usefulness of 
the course. In neither group did a majority of the students 
endorse the course sufficiently to recommend it to a friend, but 
more project than regular students were willing to advertize 
the course to other students. In the spring quarter the differ- 
ence in attitude between the students in the two groups was 
almost negligible. By far the most important difference 
between the groups for both winter and spring classes was 
brought out by the question as to which type of work the stu- 
dent would register for if he were given a choice. This question 
yields a rough measure of the student’s satisfaction with the 
work he pursued. In the winter 80 per cent of the project 
students expressed satisfaction by stating that project work 
would be their choice, whereas only 37 per cent of the regular 
students were sufficiently satisfied to choose routine work. In 
the spring when students actually were allowed to make a 
choice between project and regular work 96 per cent of the proj- 
ect students were satisfied with their choice, and again only 
38 per cent of the regular students were satisfied. 
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As a further item on the questionnaire criticisms of each type 
of course were stated as concisely as possible and the student 
was asked to record whether or not he considered the criti- 
cism valid. Both groups considered as advantages of the proj- 
ect method the opportunity for choosing a problem related to 
the student’s interests and the necessity for original and inde- 
pendent thought. Both believed that experiments in the regu- 
lar course were unrelated to their interests and considered that 
fact a disadvantage. Regular students feared that work on a 
single project for several weeks might become boring, but 
project students said this was not true. 

In the opinion of the students, then, project work has ad- 
vantages over regular work in that it is more interesting, it gives 
as much or more knowledge, it requires slightly less time, it 
helps to relate psychology to the students’ other interests, and 
in general it gives more satisfaction. 


GENERAL DISCUSSION 


For the purpose of this study it is assumed that if any one 
method of teaching experimental psychology is to be considered 
better than another from the student’s point of view it must do 
for him one or more of the following things: give him more 
knowledge of scientific methods, or teach him more facts and 
principles of psychology, or make him aware that psychology 
is a science which can, by scientific method, solve the problems 
which loom large upon his own horizon, or make him critical 
of the theories which pseudo-scientists foist upon an unwary 
public, or increase his interest and pleasure in some field of 
knowledge, or contribute to his general intellectual and, if 
psychologists may use the term, spiritual growth. 

From the instructor’s point of view the better method must 
help to keep him abreast with psychological thought and 
techniques, make his teaching more pleasant and effective, and 
must not require time out of proportion to the results achieved. 

From the points of view of both the better method must bring 
about greater coéperation and more complete rapport between 
instructor and students. 
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The first step in evaluating the two methods, therefore, is to 
ascertain how much each group has learned of facts, principles, 
and techniques. Although the objective results (tables 2 and 
3) show only slight differences, they are rather consistently in 
favor of the project method. That project students also 
learned certsin things which were not included in the examina- 
tion is evidenced by the following facts which, although they 
do not lend themselves to analysis by statistical methods, are 
as obvious as those recorded in the tables. 

1. As has been stated above several new experimental and 
statistical methods which were not included in the regular 
course were taught to the project students individually. 
Practically all project students learned at least one new method 
and several learned three or four. To be specific, all students 
who collected data on more than 50 cases learned to work 
Pearsonian correlations by the use of one or another correlation 
chart. The use of this method also made it necessary for them 
to understand guessing and correcting an average and comput- 
ing the standard deviation. That students actually knew what 
they were about was indicated by their betting on the size of 
the correlation from the appearance of the scattergram and by 
their checking frequencies from one chart to another in a long 
series of intercorrelations. 

Two or three groups learned to use the Edgerton-Paterson 
tables for computing the P.E. of percentage.’ Several learned 
to apply the multiple choice method or the rating scale technique 
in composing new tests. Some became familiar with Thurs- 
tone’s method of measuring attitudes. Others worked out a 
qualitative scoring system after tabulating frequency of re- 
sponses as in the Kent-Rosanoff free association tables. Many 
learned complicated scoring methods such as were used for the 
overstatement test or personal rating blanks. A few worked 
out very elaborate testing material with little or no assistance; 
as in the case of the architecture student who constructed a 
graded series of drawings of buildings representing varying 


* Edgerton, H. A., and Paterson, D. G.: Jour. Applied Psychol, 
1926, x, 378-391. 
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degrees of balance, and of the student of literature who worked 
out a test of prose appreciation similar to the Allen and Trabue 
Poetry Judging test. Almost all of the students learned to use 
the calculating machines or the slide rules in their computations. 
Occasional students consulted the psychological literature and 
learned to use the Psychological Index. 

The objection might be raised that the students followed the 
directions of the instructors so mechanically that they did not 
really appreciate the significance of what they were doing, but 
occasional naive remarks from the students indicated that this 
was not thecase. One group of students, who had learned the 
Pearsonian method of correlation, obtained one coefficient which 
was inconsistent with the other correlations in their series. 
They came to the instructor and said: ‘‘How do you account 
for this correlation of +.70? Our figures are right; we’ve 
checked them and we have also checked our frequencies 
against those on other charts for the same trait. And don’t 
you think the correlation looks as if it were high and positive 
from the chart? But we don’t know how to explain it!’ 
Neither, indeed, did the instructor, but she was gratified at 
their careful and understanding use of the correlation method. 

One student who, after completing his study found that his 
topic had been adequately investigated by a British psycholo- 
gist working on the G-factor, waggishly remarked in defending 
his experiment, ‘“‘Well, it didn’t matter that we didn’t read the 
British article first. After all, he came to the same conclusions 
that we did.” 

Another student in trying to explain some data wrote “It is 
possible of course that the casual reader of the Daily does not 
get as much from the advertisements as those who read it for 
the purpose of looking for sales and styles. As I was writing 
the above statement I decided that I could prove it by checking 
over my data. Here aretheresults.’’ Then follows afew lines 
of figures, and: ‘‘My conclusions were a little early. The pro- 
portion is about the same; in other words I cannot explain my 
data in those terms.” 

Another student wrote in her concluding paragraph, ‘‘Some 
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may think this method (of scoring) is too strict for a test of 
creative imagination, but it was successful in raising the correla- 
tion from .40 to .60,” and “since none of these things raised my 
correlation above .62, I couldn’t do anything else but admit 
that the test was not reliable.”’ 

2. Whether or not they were required by formal assignment 
to do so, project students often criticized their methods and 
results, suggested improvements on their experiments and 
thought out corollaries to their problems which might be solved 
by the same or similar methods. That this critical attitude 
was developed while the students were working on their experi- 
ments was shown by the difference between their interview with 
the instructor at the beginning and at the end of their experi- 
ments. They limited their problems much more definitely; 
they realized the existence and the subtle nature of chance 
errors, and they were more exacting in their control of condi- 
tions. In particular they felt the necessity of basing their 
conclusions on a large number of cases, and were eager to ar- 
range for additional cases, and for more varied groups of sub- 
jects. They became more carefully critical of all new material 
offered, largely because their own work had given them a more 
personal and definite point of view from which to evaluate 
psychological experiments. They learned more intimately the 
nature of psychology as a growing and experimental science. 
After their own experience in devising a test for some specific 
trait, in trying it out on one or more groups with high hopes 
for success, in checking it for validity and reliability, and in 
discovering their more or less total failure, they had a first 
hand knowledge of the field of testing which made them at the 
same time more critical and more tolerant. In the end the 
regular students knew as many facts, but the project students 
were wiser. 

3. For the reason that they were the actual fruits of the course 
over and above the routine work the projects themselves are 
the most objective evidence of what the students accomplished. 
Although the instructors who directed them would be the last 
persons to consider the problems of importance, or the results 
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obtained by the students of scientific value, nevertheless in 
several cases students who repeated an experiment or used a 
technique which was currently used by psychologists obtained 
results very similar to those reported in the literature. Some of 
the projects, inadequate in themselves, still served as explora- 
tory work in a new field; in some cases the problem was under- 
taken definitely with that end in view and the work of sopho- 
mores served as a basis for subsequent work to be carried on by 
an instructor or a graduate student. It seems likely since so 
much constructive work was going on in the course in addition 
to the routine work that the students were to some extent 
receiving the benefit. 

The results of the questionnaire showed rather consistently 
that project students found more interest and pleasure in their 
work and valued the opportunity it gave them for solving the 
problems which appealed to them. Again many objective 
details can be cited which add weight to these findings. After 
the student has taken part in a dozen routine experiments and 
has written eight 12-page analyses of his own and the class 
results he is far more interested in a longer experiment in his 
own field of interest than in a dozen more routine experiments, 
provided that he has the ability to originate a problem himself 
or to carry out one suggested by the instructor. In general, too, 
the range of experiments with which the project students come 
in contact is wider and, to the sophomore, more interesting and 
important. 

Students who had definitely chosen their major subjects or 
who were already established in some field of work gained most 

‘in these respects,—the music students, and the students in 
English, in architecture, and the Daily reporters. The young 
pastor who studied the relation between knowledge and atti- 
tude in religion became aware that scientific inquiry into holy 
matters is not necessarily dangerous, but that it may be ex- 
tremely enlightening. Impressed by her discoveries about 
students’ knowledge of the movies, the student who championed 
the cause of the legitimate stage went straight off to tell her 
results to the manager of the theatre in which she was an usher. 
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The negro youth who arranged the “tom-tom” experiment 
cheerfully proposed improvements of his technique in spite of 
his keen disappointment at not having hypnotized his subjects. 

Even those students whose interests did not lead them beyond 
the walls of the campus saw how psychology could attack the 
problems right at hand. The students who discovered that 
even though sorority sisters may run to type in dress and man- 
ners, they do not do so in emotional traits, and those who found 
that their fraternity brothers were not very clever at identify- 
ing cigaret brands were attacking problems of intrinsic interest 
and value to them. In short the best argument for the use of 
the project method lies in the fact that the instructor does not 
set the student to the task of re-establishing a fact that some 
psychologist of the early days has already established, a fact 
which the student has already learned from his text books, and 
about which he has no natural curiosity. Nor even is he set 
the task of finding out something which the instructor himself 
wants to know, but rather he is given access to the best possible 
methods and advice for discovering more about the particular 
psychological principles which his own interest and curiosity 
lead him to investigate. 

When the projects were reported before the class at the end 
of the course they called forth lively and intelligent discussion 
and the students met very ably the criticism of their class- 
mates. Thirteen projects which were presented by the students 
before members of the departmental seminar made a favorable 
impression on the members of the department who expressed 
their approval by interested discussion with the students and 
later by comments to the instructors. Needless to say the 
students who gave the reports were gratified and encouraged 
by the attention which they received on this occasion. 

No objective measures analogous to the examination and the 
questionnaire exist for weighing the time and energy required 
from the instructor against the interest and pleasure she found 
in the project method. Keeping twenty student pairs at work 
on twenty projects demanded somewhat greater organization 
and different distribution of her time. The necessity for ad- 
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vising students in their work kept her alert to new methods and 
techniques which she had no occasion to use in her own re- 
search and which she knew only through the literature. There 
is some saving of time with this method from the fact that the 
students hand in only one notebook to be read and graded, 
rather than eight. In actual numerical terms if an instructor 
has charge of two sections of 40 students each with two students 
working on a project the number of notebooks to be read and 
graded will be 40 rather than 640. One instructor who filled 
out a time record card every day for six weeks found that for 
every two hours spent in conference with project students and 
in supervising and criticizing their work she spent three hours in 
making out objective examinations, and in revising and arrang- 
ing for routine experiments, work which is necessary in any 
laboratory course. But it would seem that of the total time 
alloted for teaching if three-fifths must go to the mechanics of 
teaching, two-fifths are not too much to devote to the instruc- 
tion of the student, who after all is the person supposed to be 
benefited by the teaching process. 

The interest and pleasure which the instructor finds in the 
project work can be estimated only subjectively and would 
vary widely from one instructor to another. The instructors 
in this study agree that the added interest which they them- 
selves found in the project method compensated for the greater 
labor it involved. 

The third criterion for evaluating the two methods is the 
spirit of coéperation and rapport between instructor and class, 
and here again the evidence can be produced only from subjec- 
tive estimates based as far as possible on objective details. In 
the routine course the presentation of material by the instruc- 
tor is more formal, more nearly like the presentation of material 
in textbooks. The project method introduces, by means of 
its stronger appeal to the interest and ability of the student, a 
new element of frankness and equality in the relation between 
the student and his guide in the acquisition of knowledge. 
Either the instructor or the student may resent the introduction 
of this new element. The instructor may prefer the more 
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formal presentation of his subject, or he may be convinced of 
the greater value of a routine series of experiments as compared 
to individual projects. Likewise the student may prefer the 
more formal course work to original and individual work. 

In the present study both instructors believed that the 
necessary change in attitude was a desirable one. Both in- 
structors remember many interviews with project students in 
which they gave expression to their interest most enthusiastic- 
ally, and likewise both remember some unpleasant conversa- 
tions with regular students who complained at length of the 
dullness of the course and the inordinate amount of time re- 
quired to write notes on experiments. Although the project 
students may have felt dissatisfaction they did not rehearse 
these long complaints probably because they were working on a 
problem of their own choosing and because the instructor was 
obviously giving time and attention toward making the project 
a success. 

The only difficulty in the attitude of the class toward the 
instructor came at the beginning of the project work when the 
problems were being arranged for individual groups. There 
were always some students who were unable to originate a 
problem and who were helpless and even timid about indicating 
a field of interest. The prospect of doing anything without a 
textbook and printed directions bewildered them and they 
apprehensively resented the change in the teaching method. 
Likewise some students did not at once find a problem suitable 
for them. One such student went through a half-hour inter- 
view calmly and firmly turning down every proposition the 
instructor made. “No, I’m not interested in that problem.”’ 
“No, I wouldn’t want to join that group. I don’t know any 
of those girls.” This type, since he had ideas of his own, 
usually did good work once a problem had been arranged for 
him. The former type remained a willing, but a sad and in- 
adequate worker to the end of the course. 


GENERAL APPLICABILITY OF THE PROJECT METHOD 


At this point it is well to consider the criticism that project 
work is not suited to the entire range of ability of sophomore 
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students. From the foregoing account it may seem that the 
method is better adapted to the needs of superior students 
than to those of average or inferior ability. Bearing in mind 
that no course is equally well adapted to the good student and 
the poor, the interested student and the bored, the worker and 
the drone, the instructors still feel that the project method has 
general applicability. 

The correlations presented in table 4 indicate that the rela- 
tionship between final exam grade and college ability is quite 
low for both project and regular students and that the relation 
of exam grades for the two halves of the course is about the 
same for both types of students. The letter grade of about 
half the students in each group changed from one quarter to 
the next; a larger per cent of the project than regular students 
changed to a higher grade for the second half of the course, 
and a larger per cent of regular students changed to a lower 
grade the second half. In view of the facts presented in table 
4 the suitability of project work for students of each level of 
ability in the grade range will be considered. 

A. The superior student. The good student, one whose 
grades in college average A or B, whether interested in the 
course itself or merely obtaining a high grade in it, does good 
work in either type of course. In the regular laboratory he 
performs his experiments with care, reads the suggested refer- 
ences, and writes intelligent reports. In the project section he 
apparently likes the opportunity for doing original thinking 
and he works independently, consults the literature, collects 
data systematically and works it up in an able manner.. With 
such students any teaching method is successful. 

B. The student of medium ability. In every class there is a 
large number of students who do steady, consistent work of a 
mediocre level. Their efforts may be great and their achieve- 
ments are not to be despised, but they almost never show 
flashes of brilliancy, nor rise to the point of criticism. A method 
of teaching which will raise the level of work of these students 
is indeed hard to find. In the regular course they do a large 
number of experiments with care and write up their reports at 
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length, but it is often apparent that they have only an inkling of 
the nature of the problem and the significance of the results. 
Since in the project method their attention is narrowed down 
to one experiment, it seems probable that they get a clearer 
notion of the problem, the methods and the results than they 
get from rapidly skipping through a number of experiments. 

C. The inferior student. Neither the regular nor the project 
laboratory is planned to fit the needs of the inferior student. 
If he is to pass the course at all he must “ride” through, aided 
by the more or less invisible efforts of others. The opportuni- 
ties for “riding’”’ are about as great in one type of course as in 
the other. But “riding” is perhaps less easily concealed in the 
project section, since it has to be done either on the back of the 
partner in the experiment, or on that of the instructor; whereas 
in the regular section the “rider’’ may choose his “mount” from 
the same class, from another laboratory section, from his 
fraternity, or from the accumulated stores of notebooks about 
the campus. 

D. The discrepancy student. If the project method has an 
advantage over the regular method for any type of student it 
is for the “discrepancy”’ student,—the student who has better 
than average ability, but who will not settle down to routine 
laboratory work because it is distasteful to him. Often such 
students give themselves excellent reasons for refusing to work. 
Some of them are true conscientious objectors to the system of 
cut-and-dried course work and the rigid completion of require- 
ments which exists in most American universities. Others 
are thoroughly convinced that the training they get from report- 
ing on the Daily, starring in a campus play, or dominating class 
polities is of greater value than any accumulation of knowledge of 
English, political science, or psychology. And finally some are 
students who have come into experimental psychology with the 
impression that here they will attack in a scientific way prob- 
lems which are pertinent to them in their intellectual, emotional, 
and social adjustments, only to find that the course has to do 
with reflexes, card sorting, and speed of reading. Every 
laboratory class has such students on its roll, and for those 
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students the routine course is a failure. To get them through 
it at all the instructor has to prod constantly, with the reminder 
that it is better to pass an unexciting or boring course the first 
time than to flunk and have to go through it again. To such 
students the project method gives adequate motivation and 
incentive to good work. The conscientious objectors are grati- 
fied at getting their teeth into an original piece of work, and at 
being relieved of some of the burden of rigid assignments. The 
campus politicians discover ‘that psychology is not unrelated 
to the important business of life which is exemplified by their 
hundred and one “‘activities.’’ And the students who registered 
for the course because they wanted to learn psychology find that 
the solution of some intimate problems are within their powers. 

Needless to say, the instructor must use the greatest insight 
and tact in assigning these discrepancy students projects com- 
patible with their other interests. Once started on the right 
track they usually require no more supervision than the other 
groups. Often when they are thus motivated they do an ex- 
ceptionally good piece of work. The instructor no longer has 
to justify and defend the course to them; and now and then 
through the project work a student “finds” himself to the 
extent that he establishes interests and habits of work which 
last throughout his college years. A teaching method which is 
highly successful with the disinterested students is doubtless 
equally good for the interested. 


SUGGESTED PROGRAM FOR THE LABORATORY COURSE 


In planning a program for the laboratory course at Minne- 
sota the experiences of the instructors during the spring term 
particularly were taken into account. It will be remembered 
that the objective differences between project and regular 
students were much smaller in the spring than in the winter. 
(Tables 1, 2,3 and 4.) This was to be expected in view of the 
fact that since the regular and project students in the spring 
were all in the same sections each knew more about the work of 
the other group. Project students often took part in the routine 
experiments, although they did not write accounts of them. 
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Regular students acted as subjects for the projects, listened to 
reports of them, and received all the benefits of the class dis- 
cussion. Whenever possible the instructor used these experi- 
ments and discussion as teaching material. Since only the 
students who expressly preferred the project work took a part 
in it most of the difficulties due to lack of initiative and ability 
to originate or carry out a problem were eliminated. These 
advantages were offset to some extent by the difficulty of carry- 
ing on both routine and project experiments with the same 
groups of students. 

On the whole this combination of the two methods was con- 
sidered the most successful, and it has been adopted for use 
during the present year. Routine work will be carried on 
during both halves of the course, but during the second half a 
few more or less self-selected student groups will arrange and 
carry out projects which will supplant all or part of their 
routine work. The projects will be selected by the instructor 
with an eye to their usefulness for general class instruction, and 
they will be definitely incorporated into the course. 

Other programs might be suggested, but the most successful 
plan would always take into consideration the interests and 
abilities of the student and the wishes of the instructor. The 
occasional student who really dislikes to do original work and 
who undertakes a project with misgivings should not be forced 
into project work. And an instructor who is convinced of the 
value of routine drill or who likes to have his course developin 
a logical and orderly manner or who feels that sophomore 
students are too young to be given a free rein probably would 
not find the project method a success. 

To conclude, the project method as it was carried out at 
Minnesota is better than the routine method of teaching psy- 
chology laboratory only in that it consults the students’ interests 
and it gives the majority of them greater satisfaction. Psy- 
chology as one of the younger sciences is still in the stage where 
it must take account of its standing in public opinion, for like 
other sciences it is dependent upon public attitude for support. 
Satisfied students who have acquired the idea that psychology 
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actually can and does solve problems of personal interest to 


them are the best guarantee of support for psychology in the 
future. 


CONCLUSIONS 


1. Students taught by the project method learn as much as 
regular students when their work is measured by the ordinary 
“objective” examination for the laboratory course. 

2. According to available objective evidence the project 
students derive as much interest and satisfaction from their 
work as the regular students. 

3. Since the project method required proportionally more 
time from the instructor in the course and less time from the 
assistants and readers of notebooks, especially competent 
assistants can take over part of the additional burden. 

4. The method is somewhat more successful when students 
who have a decided preference for original work are allowed to 
elect project work and those who prefer no change in their work 
are kept occupied with the regular experiments. 

5. Experience with students of different ability seems to 
indicate that: 

a. For the superior students either method is successful. 

b. For the “average” student one method is as good as 
another. 

c. For the inferior student both methods fail. 

d. For the “discrepancy” student the project method, which 
motivates him to good work, is very much the better method. 





A METHOD FOR MEASURING MUSICAL TASTE 


P. E. VERNON 
Yale University 


From an impartial psychological viewpoint the only satis- 
factory definition of an individual’s musical taste would seem 
to be the degree of approximation of his musical likes and dis- 
likes to the opinions of a consensus of musical leaders, conduc- 
tors and critics, of the time. In other words, if he appreciates 
music which is commonly acknowledged to be great he is said 
to have good taste. This greatness of a musical composition 
is, of course, relative to the musical standards of the particular 
time and place; we have no right to consider it as objective and 
universal, since so many affective and irrational suggestive 
factors determine it. Hence, also, no one’s taste can be 
regarded as absolutely better or worse than another’s; it is called 
good or bad in so far as it agrees with or differs from the views 
of the average trained musician of the same country, culture and 
period. 

The obvious way to measure this characteristic would be to 
play several musical items to our individual S, items which are 
generally considered as good or bad, and compare his account 
of his appreciation with their accepted value. There are sev- 
eral strong disadvantages to such a method. 

1. To cover the ground at all adequately an immense number 
of musical items would be needed. So many uncontrollable 
factors would condition S8’s appreciation of, say, a thirty min- 
utes concert that we might expect the statistical split half 
reliability of the method to be extremely low. 

2. Every hearing of a musical item is different from every 
other. In an intelligence test we insist on common educational 
opportunities, social status and nationality among our sub- 


355 











356 P. E. VERNON 


jects, or else we get paradoxical results such as Gordon’s with 
canal boat children. Similarly we ought to try to stand- 
ardize in such a musical test the effects of racial prejudices or 
of topographical or financial conditions that may have pre- 
vented S from hearing the usual amount and kind of music, a 
very difficult thing to do. But in addition his musical appre- 
ciation, far more than his performance at an intelligence test, 
will be influenced by ephemeral factors such as the chair he is 
sitting on, the dinner he has eaten, the time of day and his 
personal relations with the experimenter. Thus the repeat 
reliability would also be very poor. 

3. The mere fact of having to record some judgment would 
tend to hinder him from appreciating normally.! Unless 
specially devised concerts were employed the phonograph 
would exert further distortions. 

4. If S knows anything at all about music he is certain to 
have developed fixed opinions. He will probably know that 
Bach and Beethoven are generally considered great composers, 
Gounod and Ketelbey poor ones, even though his personal 
preferences may lie in the opposite direction. To perform the 
music anonymously with such an S would play havoc with his 
responses, since he might recognize some of it or guess the com- 
poser, correctly or incorrectly. In fact he cannot help but 
appreciate to a large extent as he thinks he ought to appreciate 
when any judgment is required of him by an experimenter. 
Only if he is totally devoid of musical knowledge and experience 
would such a method work; and in such a state we might find 
that his musical taste also approached zero. 

I was myself, however, more interested in ordinary music 
lovers and ‘amateur musicians who go to concerts rather fre- 
quently and are already familiar with a large quantity of music. 
To overcome the first three difficulties cited above it seemed to 
me essential to apply some more generalized method which, 
while blurring the details of taste, would give a more adequate 


1 The difficulties of attaining approximately normal concert condi- 
tions in psychological experimentation are discussed in my article: 
Method in Musical Psychology, Amer. Jour. Psych., xli, 1930, 127-134. 
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survey of the whole field. The plan adopted, whose technique 
is somewhat analogous to that of pencil and paper Ethical 
Discrimination tests, brings in effect about sixty hours of varied 
musical situations before the S upon which within twenty 
minutes he reeords his appreciation. The instructions given 
to subjects will explain it; they were as follows: 


CHOICE OF PROGRAMMES 


Below are a number of programmes of specimen concerts to be given in 
the usual concert halls. You are asked to decide how much you would 
like to go to each of them, supposing that you would be given reserved 
seats for those that you chose. Assign marks to each one according to 
the degree in which you would like to go. 

The maximum mark, 4, means “J should very much like to go to this 
one.’’ 

The minimum mark, 0, means ‘J do notin the least want to go to this 
one.”’ 

3, 2 and 1 are intermediate grades. 

In addition to these marks you are particularly requested to make any 
qualifications or modifications you like; for instance, always say if you 
are influenced by special pieces or performers. 

Try to indicate your real tastes by your choice, not merely what you 
think you ought to go to. (The programmes are in haphazard order so 
as to give no indication of the goodness or badness of any one by its 
position on the list.) 


Then followed thirty imaginary concert programmes, to- 
gether with the names of the performers. I compiled these 
largely from actual programmes, from victrola record catalogs 
and radio announcements, etc. I shall not reproduce them here 
since they were meant to refer specifically to the common 
musical experiences of my subjects, namely English under- 
graduates and graduates. They could not be applied in a com- 
munity that had experienced unlike musical opportunities. 
They differed from the usual concert programme in that each 
one was made as homogeneous as possible in itself, and was 
representative of a different type of music from the others. Full 
orchestral concerts, chamber music, choral concerts, operas, 
solo recitals, concert parties, military bands, etc., were repre- 
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sented. The answers were given to me verbally and individ- 
ually so that many details of a qualitative nature were noted 
such as 8’s attention to performers rather than music, or vice 
versa, his preference for familiar or unfamiliar music, his 
capacity for inferring from known works of a composer to 
unknown works, special prejudices that did not show up in the 
general mark given to the concert as a whole, and so on. 

The drawbacks to this method are obvious. The chief one 
is that it can only be employed with people who are sufficiently 
musical to know a lot of music by name. Orif they have not 
actually heard and cannot recall their impressions of a piece of 
music they must have sufficient general musical knowledge to 
be able to say what their opinions would be. Actually it was 
very seldom that my 8’s found themselves quite unable to give 
a definite judgment. The method works well with educated 
people of this type but could not be extended to the relatively 
unmusical. 

At first sight the test might appear to suffer even more than 
the plan which I condemned above from the fourth disadvan- 
tage, namely, that S’s would tend to answer what they thought 
they ought to answer. But it seemed to me better to go to 
meet the difficulty rather than to pretend to ignore it. Hence 
the directions specifically warned S’s against this error. The 
mere facts of being somewhat removed from the concrete con- 
cert conditions and unreliable emotions (especially from ab- 
normal experimental conditions) to the sphere of sane and 
normal discussion and specific intended behavior add sincerity 
to their judgments. They were quite unaware of the subse- 
quent interpretation of these judgments in terms of quanti- 
tative levels of taste. Again, since the judgments were told 
to me in person, it was usually possible to tell from the addi- 
tional, qualitative remarks whether each S was really giving 
his own opinions or artificial opinions which he or she thought 
to be correct. The very great variety of marks given to all 
the concerts (except the very worst ones) proved that this 
distorting factor was rather successfully eliminated. 





METHOD FOR MEASURING MUSICAL TASTE 


SCORING 


The standard against which the opinions of 26 S’s were com- 
pared was based on the opinions of six of my most musical 
subjects, obtained under the same conditions as applied to the 
testees. A digression is necessary into the determination of 
musical-ness or unmusical-ness, ‘‘musical level’? as I prefer 
to call it, corresponding to the German Musikalitat. 

In this and other musico-psychological investigations I used 
104 subjects. No raters were available who knew all of them, 
but ten of these S’s were chosen as being well acquainted with 
the greatest number, and an average of more than four 
ratings per subject were thus obtained, on a 0 to 10 scale. 
These raters were given the following definition: 


By musical level is meant a combination of the following qualities: 

a. Interest in music; e.g., amount of time spent on musical pursuits 
spontaneously and voluntarily, but not necessarily mere emotional 
sensitivity to music. 

b. Knowledge of musical matters. 

c. Goodness of ‘‘ear’’ for music; e.g., sense of pitch and rhythm, 
musical memory, etc. though not necessarily capacity for performing on 
instruments. 


Owing to the incomplete overlapping of the ratings it is 
difficult to work out an exact reliability coefficient, but the 
average correlation of any one set of ratings with the final 
rank order was .90 + .021. This shows that musical level is a 
fairly definite characteristic, its criteria are based largely on 
actual behavior rather than subjective intuition. Such ratings 
might well be verified by giving an examination or test on all 
aspects of music to the S’s. Performance at such a test would 
depend on interest and talent as well as education so that a 
very high agreement might be expected.” 


? The Kwalmasser Test of Musical Information and Appreciation 
could not have been applied in its present form since many of its ques- 
tions and its norms are specifically American. But from a few observa- 
tions I find that a P.R. of 100 (a score of 210 out of 250) corresponds 
roughly to a musical level of 8 out of 10 on my scale, and a P.R. of 50 toa 
musical level of 14 or 2. It therefore follows, incidentally, that my 
measure of musical taste is in general unsuitable for those who score 
below the average on such a test as Kwalmasser’s. 
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Taking, then, the average mark given to each programme by 
the six most musical subjects available, a standard score was 
empirically found for the programme. The summed devia- 
tions of the actual marks given by an S from these standards 
were used as an indication of this S’s taste. Thus the standard 
mark: for the first concert was 2; S’s who gave it 1 or 3 marks 
scored a deviation of 1, those who gave it 0 or 4, a deviation of 
2. (Halves were also allowed, but no smaller fractions). The 
total possible deviations on all the concerts was 92; this in- 
cludes deviations of 4, 3}, 3, 23 and 2 in approximately equal 
numbers. An §’s actual taste marks are expressed as a per- 
centage of this figure. Actual deviations ranged from 17 to 61 
points. (Naturally the six S’s who were used as standards 
obtained lower total deviations, but 17 was the lowest among 
S’s who were used as testees.) These therefore correspond to 
percentage scores of 81} and 34 respectively. 


VALIDATION 


Now this method defines taste almost Behavioristically in 
terms of the opinions of the most musical. It is an empirical 
measure for which no further subjective nor functional sig- 
nificance is claimed. As long as the people who were used as 
standards were of the same general culture, education, etc., 
as the testees I was satisfied with this quantitative estimate of 
tastes relative to people of this type. Hence the best kind of 
validation from this objective point of view is that supplied by 
the reliability or internal consistency of the method. Does the 
test measure reliably the characteristic that it sets out to 
measure, irrespective of what the inner nature of that charac- 
teristic may be? The corrected split half reliability coefficient 
between equivalent paired items of the test was .846 +.035 
(Spearman rank correlation). No doubt the programmes might 
be considerably improved in their construction so as to give an 
even higher figure, but apart from this we can predict sta- 
tistically that a test of only three times the length would be 
needed to give almost perfect reliability and consistency (a 
coefficient of .95) for the method. 
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To compare the results with subjective ratings for musical 
taste would be futile on a priori grounds. Probably the 
reliability of such ratings would be far lower than that of the 
test since ‘reputation’® for musical taste would have even less 
basis in objective behavior than, say, sense of humour or un- 
selfishness. But a further check of the validity both of the 
test and the initial assumption on which it was based is afforded 
by comparing taste marks with musical level ratings. If it is 
true that good musical taste is the approximation to the tastes 
of the most musical, then one would expect a high, though not 
a perfect agreement between the musicalness of the individual 
and his tastes. The raw Spearman rank correlation between 
these two sets of estimates for 26 S’s is .70; corrected for atten- 
uation this becomes .76 +.058. (Naturally the six musicians 
who were used as standards are not included among the testees 
in this calculation.) The small number of subjects is offset 
by the high value of this coefficient. 

It seems then that these figures go far to justify the further 
use of this type of method for the investigation and estimation 
of musical taste in any community consisting of moderately to 
very musical people. On examining the measures qualitatively 
I found that any considerable differences between rank positions 
for taste marks and musical level could almost always be attrib- 
uted to abnormalities in musical upbringing, or distorting 
psychological factors such as temperamental biases, contra- 
suggestibility and external interests. That is to say, the 
agreement would have probably been even higher if this cri- 
terion of equality of musical opportunity had applied more 
perfectly. 


SUMMARY 


1. Musical taste is considered objectively as the approxi- 
mation of musical opinions to the opinions of musicians of the 
same culture, race and period. 


*Cf. H. Hartshorne and M. A. May: Studies in Service and Self 
Control, 1929. 
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2. It was estimated among musical amateurs by comparing 
their preferences for a series of thirty imaginary concert 
programmes with the averaged preferences of six highly musical 
subjects. The musicalness or “musical level’’ of such subjects 
was established by ratings whose reliability averaged .90. 

3. The split half reliability or internal consistency of this 
pencil and paper method was .846, a much higher figure than 
could be expected from judgments on actual musical selections 
played to the testees, or from ratings on musical taste. 

4. The taste marks correlate with musical level ratings to 
.76, thus validating the above assumption on which they are 
based. 


SOME SALES RESULTS FOR POSITIVE AND NEGA- 
TIVE ADVERTISEMENTS 
D. B. LUCAS 
Rutgers University! 
AND 
C. E. BENSON 
New York University 


The following sales results for positive and negative adver- 
tisements were obtained in the course of coupon and recall com- 
parisons previously reported.” It is difficult to obtain accurate 
sales results for advertisements, except where the sales are of 
the mail order type. Sales values are the actual criterion of 
the effectiveness of appeals. Only the impracticability of 
securing data prevented the building up of a comprehensive 


analysis of positive and negative appeals based on sales records. 
The material included in this report serves to substantiate the 
claim of negative appeal to a place in modern, scientific adver- 
tising. ; 

The comparisons which follow include sales records for exten- 
sion educational courses, a test campaign, and an unusual 
experience in promoting a book. The heterogeneous nature of 
the material is compensated by the fact that sales are the 
measure of effect, and that the data apply directly to the 
problem. The use of various methods of gauging sales adds 
further interest to the results. They suggest techniques for 


1 Now at Iowa State College. 

* Lucas, D. B., and Benson, C. E., The Relative Values of Positive 
and Negative Advertising Appeals as Measured by Coupons Returned. 
The Journal of Applied Psychology, June, 1929, Vol.. XIII, No. 3. 

Lucas, D. B., and Benson, C. E., The Recall Values of Positive and 
Negative Advertising Appeals. The Journal of Applied Psychology, 
June, 1930, Vol. XIV, No. 3. 
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future studies in advertising, and especially for comparing the 
merits of positive and negative appeals. This question has 
been the center of much argument, both academic and in busi- 
ness. Thus far, in their studies, the authors have found little 
evidence against negative advertising appeals as they are now 
used. 


DEFINITION 


Positive advertising appeals are those which urge the reader 
to the attainment of something desirable. Negative appeals 
place the emphasis upon annoying or repulsive situations and 
point the way to their avoidance. Not all appeals, even though 
personal, are distinctly or strongly positive or negative. For 
that reason the study has been confined to the more outstanding 
specimens of each type. 


THE PROCEDURE FOR EDUCATIONAL COURSES 


The entire procedure for comparing sales was similar to that 
used in the earlier coupon analysis. Advertisements of each 
type were selected and comparisons were made between indi- 
vidual pairs under equally favorable circumstances. Each 
advertisement of the pair represented the same product, and 
was similar in size, the use of color, illustrations, headlines and 
text. In some cases where the cost per sale was used for com- 
parison, the average cost for several insertions was used because 
of the added weight. The average cost per sale for one of the 
advertisements which was run 69 times is probably of more 
value than a figure for a single insertion. It has been found 
that costs per sale (or coupon) and total numbers of sales (or 
coupons) are interchangeable in comparative studies. Since 
the advertisements used in the pairs were equal in objective 
aspects, they were practically equal in cost. As a result the 
relative values of the total number of sales per advertisement 
correspond closely with the relative costs per sale. The adver- 


* Lucas, D. B., and Benson, C. E., The Relative Values of Positive 
and Negative Advertising Appeals as Measured by Coupons Returned. 
The Journal of Applied Psychology, June, 1929, Vol. XIII, No. 3. 
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tisement getting the most sales would obviously get them at the 
lowest cost per sale. 

Correspondence schools and similar institutions were found 
to be about the only source from which definite numerical 
results in sales could be obtained. Their names have been 
omitted here because the figures are confidential with respect 
to the institution, the publisher and the advertising agency. 
These institutions receive inquiries in response to their adver- 
tisements. The contacts are followed by personal letters and 
eventually a certain percentage of the inquiries result in sales. 
It is the experience of several companies that advertisements 
can be designed which will bring in a volume of requests for 
additional information, but which result in few sales. It ap- 
pears that the quality of the advertisement has some effect 
upon the quality of response received as well as the quantity 
or number of replies. The advertisement is due a certain 
amount of credit for the sales it brings, besides the mere number 
of inquiries. 

The sales results for each advertisement in each pair were 
originally recorded on the same line. The relative value was 
then computed by dividing the negative score by the positive 
score, and translating into per cent. If the scores were equal 
the relative value would be 100 per cent. If the positive value 
exceeded the negative the result would be less than 100. If the 
negative value were greater the relative value would be above 
100. When numbers of sales were used as a basis, the relative 
value was found directly by dividing the negative score by the 
positive. In the case of costs per sale, it was necessary to 
divide the positive cost by the negative to arrive at a corre- 
sponding value. It will be clear that a higher cost per sale 
indicates a lower effectiveness than a low cost, whereas a high 
number of sales is better than alow number. All of the relative 
values represent the standing of the negative with reference to 
the positive. Relative values based on cost per sale are inter- 
changeable with values based on numbers of sales in this study 
for reasons already pointed out. 

Table 1 shows the results for the positive and negative adver- 








TABLE 1 





Comparisons of sales results for positive and negative appeals in advertising 
educational courses 





SALES RETURNS FROM ADVERTISEMENT 

















NUMBER RELATIVE PER CENT 
Positive Negative 

Number of sales N/P 
1 408 147 36 
2 102 132 129 
3 32 31 97 
4 33 23 70 
Cost per sale P/N 
5 $12.39 $12.52 99 
6 11.47 11.88 97 
7 14.78 16.36 90 
8 10.53 16.04 66 
9 14.83 13.95 106 
10 22.05 16.96 130 
11 26.45 14.17 187 
12 16.96 14.00 121 
13 10.28 20.85 49 
14 17.15 18.98 90 
15 12.90 24.70 52 
16 11.76 16.10 73 
17 11.47 10.18 113 
18 17.61 9.91 178 
19 8.45 16.92 50 
20 21.23 18.59 114 
21 9.36 16.33 57 
22 38.00 11.69 325 
23 19.10 14.58 131 
24 21,24 17.06 125 
25 11.21 12.56 89 
26 13.92 22.15 63 
27 5.85 6.36 92 
28 7.71 ee 108 














Summary of comparisons of sales 





Number favoring positive appeal 
Number favoring negative appeal 
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tisements for educational courses. The first three columns are 
explained by the headings. The last column carries the relative 
value for individual pairs, computed as described above. The 
summary at the end of the table shows the comparative stand- 
ings of the positive and negative comparisons. The sum- 
marized score for positive appeals is the total number of relative 
values below 100 per cent. The score for negative appeals is 
the total number of relative values above 100 per cent. 


CONCLUSIONS FROM THE DATA 


In the tabulated data there are 16 comparisons favoring 
positive appeals and 12 comparisons favoring negative appeals. 
The logical conclusion from these results would seem to be that 
there is a small, though not an outstanding advantage in 
favor of positive appeal advertising for extension educational 
courses. To determine whether a real difference exists in the 
relative strengths of the two appeals for advertising of this kind 
it would be necessary to increase the data. From the present 
data the existence of a significant difference seems doubtful. 


A TEST CAMPAIGN FOR A PROPRIETARY MEDICINE 


Advertisements are sometimes tested out in small, limited areas 
before they are released for general circulation or are discarded. 
This procedure is called a test campaign. In such a test the 
advertisement is circulated in local mediums, usually news- 
papers and the fluctuation of sales within the territory is taken 
as an indication of the effectiveness of the advertisement. If 
the sales increase enough to justify further use of the advertise- 
ment it may be placed in mediums of wider circulation. If the 
local test indicates weakness in the advertisement it is usually 
discarded or revised. 

The following experience of the manufacturer of a proprietary 
medicine is applicable to this study and especially in view of the 
numerical results which were obtained. Fifteen eastern cities 
of about equal size, and representing three sales territories, were 
selected and were divided into five groups of three each. Five 
advertisements of various types of appeals were then circulated, 
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one in each of the groups of three cities. The different types of 
appeals were as follows: 


1. Good Health, a positive appeal to the attainment of good health, 
and its preservation, through the use of the product advertised. 

2. Old and New, a reasoning type of appeal based on the qualities of 
the medicine. 

3. Dialogue, a dialogue discussion of a reasoning type based on the 
qualities of the medicine. 

4. Display, a display type of copy centered on the package and some 
of the qualities of the product. 

5. Warning, a strong negative type of appeal with emphasis on the 
prevention of ill health. 


The advertising campaign lasted four months through the 
summer of 1920 in the 15 cities, each group continuing with the 
same advertisement as it started or advertisements of the same 
type. The sales in each of the cities were then compared with 
the sales for the corresponding four months of the previous 
year, 1919. The following figures show the losses and gains for 
the five types of appeals: 


Gain Loss 
per cent per cent 
Prem DA Bs i ESSE de HO CIARA 171 
that. ite Otel he bee h ak keen s o> acaegue 115 
ER ace cu ceo tds chide gher Cet en tees en ecg sees 94 
a a ctotr renab tee sas beateobecosdvewdhde’ 80 
Good Health (positive)............c0eeeceesereseees 10 


The reason for this manufacturer adopting the warning or 
negative type of advertising for general use is apparent in 
the data. Good health appeals met with so little success that 
they were discarded for the most part. Warning against ill 
health brought an increase of 171 per cent in customers and 
sales. Before the advertisements were sent out, advertising 
experts had ranked the five pieces of copy with the positive 
health appeal as best and the negative warning appeal as 
poorest. The error in their judgment is a further proof of the 
need for more scientific knowledge about advertising, and also 
the necessity for making separate studies for each class of goods. 

This test campaign included nearly all of the essentials of an 
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ideal test of advertising. Every effort was made to give equal 
opportunities to each type of advertisement to the extent of 
controlling the territories within which they were circulated. 
The cost of this campaign approximated $60,000. That is an 
indication of the enormous outlay of money which advertisers 
are willing to make in order to improve their practice. It is also 
an indication of the handicaps which limit the efforts of indi- 
viduals who study within the field of practical advertising. 


THE SALE OF A BOOK BY MAIL ORDER 


Books are frequently sold through mail orders. The money 
may be paid on approval or, if paid in advance, it may be 
refunded on the return of the book. By making proper 
deductions where sales are not completed the results for mail 
order advertisements may be determined. The experience of an 
agency for a widely sold book was as follows: 

The book, which is intended for general home reading, was 
first advertised by a display type of appeal. The book was 
described and illustrated without reference to the reader. The 
resulting sales cost so much that no profit was left for the 
distributer. The campaign featuring display was then intensi- 
fied and a great deal of attention was given to the merits of the 
author as well as the book itself. The results were still far from 
being satisfactory. The agency then turned its attention to 
the customer’s personal affairs and started a campaign designed 
to bring the reader’s reputation into the discussion. In some 
cases emphasis was given to the success that might accompany 
a familiarity with the contents of the book. In other cases the 
appeal was made to the avoidance of inferior social standing 
which might result from ignorance of its contents. The success 
of this type of advertising was very gratifying to the agency. 
Not only did they exceed all expectations in sales but they were 
able to use the same advertisement many times before sales 
would fall off. The positive type of success appeal brought 
good results but the negative appeal involving social inferiority 
was much better. 

Two of the negative advertisements stood out as best. 
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Each brought more than twice the results, or had a relative value 
of more than 200 per cent, as compared with any positive appeal 
or display advertisement that the agency could devise. It was 
very evident for the type of product sold in this illustration 
that a particular kind of negative appeal was most effective. 
This suggests that the specific kind of appeal is of more impor- 
tance than the general type under which it may fall. 

Display appeals were ineffective in this case and positive 
appeals were quite satisfactory. In the test campaign for 
proprietary medicine the positive appeals lost patronage while 
the display advertisements brought a 115 per cent increase. 
This contradictory experience again emphasizes the need for 
separate study of individual products. 


CONCLUSIONS FROM THE STUDY OF ACTUAL SALES RESULTING 
FROM ADVERTISEMENTS 


The following points stand out from this study, based on the 
limited data: 

1. Positive appeals may have a slight advantage in advertis- 
ing extension educational courses. 

2. Negative appeal advertising seems most effective in selling 
proprietary medicines, and possibly adults react more strongly 
to negative health appeals than to positive health appeals. 

3. Personal advertising appeals, especially of a negative type, 
seem most effective in selling books for self improvement by 
mail order. 

4. Any generalities as to the relative values of positive and 
negative appeals in advertising must be supplemented by 
specific applications. 








THE CORRELATIONS OF NON-LANGUAGE TESTS 
WITH EACH OTHER, WITH SCHOOL ACHIEVE- 
MENT, AND WITH TEACHERS’ JUDGMENTS OF 
THE INTELLIGENCE OF CHILDREN IN A SCHOOL 
FOR THE DEAF! 


ANDREW W. BROWN 


Institute for Juvenile Research, Chicago 


I. INTRODUCTION 


In clinical work non-verbal mental tests are used as a matter 
of routine on all cases that have difficulty with the English 
language. They are valuable instruments. They do differen- 
tiate. But the psychologists are often hard pressed for an 
answer as to what they measure. To what degree do they 


measure success in school achievement? To what degree do 
they measure ability to get along in the ordinary occupations 
of life? Do the various non-language tests measure the same 
thing? Do they measure a special type of ability or do they 
measure general intelligence? These are some of the questions 
frequently asked. It is the purpose of this investigation to 
attempt an answer to some of them. The best way to study 
these questions is to see how closely the test results correlate 
with the best measure of the other variables one can secure. 


II. METHOD OF SECURING DATA 


The data for this study were secured from the results of a 
survey of the Jacksonville School for the Deaf, Jacksonville, 
Illinois, The survey was conducted by psychologists from the 
Institute for Juvenile Research. During the survey the entire 
school population was given individual and group non-verbal 


1 Studies from the Institute for Juvenile Research, Chicago, Herman 
M. Adler, M.D., Director. Series C, No. 156. 
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mental tests and also a battery of educational tests. Three 
hundred and ninety children were examined. For the cor- 
relation between the different non-language tests the scores of 
the entire group were used. For the correlation of the non- 
language tests with school grades and scores on educational 
tests only the scores of the children above Grade V were used. 
There were about one hundred children in these upper grades. 
The chronological ages ranged from 14 to 25 years. The 
average was about 16 years. 

All the children were given the following tests: (1) The 
Pintner Non-Language Mental Test which is especially adapted 
for deaf children and can be given entirely by pantomime. 
(2) A series of performance tests from the Pintner-Paterson 
group. Those tests which were constructed into a Point Scale 
and standardized by Grace Arthur? were used. This series of 
tests includes the Knox Cubes, Seguin Form Board, the Two 
Figure Form Board, the Casuist Form Board, Manikin and 
Feature Profile, Mare and Foal, Healy Picture Completion I 
and the Kohs Block Designs. (3) The Stanford Achievement 
Arithmetic Tests (computation and reasoning). (4) The 
Stanford Achievement Reading Tests. The Performance Tests 
were given individually. The other tests were given in groups 
of about twenty-five each. The directions for the educational 
tests were given in the sign language by the principal in the 
presence of the examiner. 

The performance tests as standardized by Miss Arthur are 
so arranged and weighted as to yield a composite score. It is 
this composite rating that is used here. 

The school marks of each individual in language and arith- 
metic were secured for each semester over a four-year period. 
This gave 16 semester grades in each subject. These grades or 
“teachers’ marks’ were averaged to secure a composite 
Teacher’s Rating for each individual in each of these subjects. 
The grades in each subject were averaged for a composite grade 
rating. 

* Grace Arthur: A New Point Performance Scale. Jr. of App. Psych., 
Vol. IX, 1925, pp. 390-416. 
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An attempt was also made to get the teacher’s estimate of the 
ability of each child in comparison with other children in the 
school. The following instructions were placed on a rating 
blank and explained to the teachers by the psychologist in 
charge of the survey: 


‘Think of the intelligence (not school ability but native ability) of 
each pupil in your class in relation to the other pupils in the school. 
Then think of a scale ranging from 1 to 20. Give each child a rating on 
this scale. If you think he is the brightest pupil in this school give him 
20. If you consider him the dullest pupil give him0. If you think he is 
about average give him 10. Place your ratings in the first column on 
this sheet indicated as Teachers’ Rating.”’ 


It was explained that the ratings should be distributed some- 
what symmetrically, according to the normal distribution. 
The teachers gave these ratings a good deal of careful con- 
sideration and the writer believes they are as accurate as ratings 
of this kind usually are. The principal and supervisor rated 
the entire group. The teachers rated only their own grade. 
Each child received at least 3 ratings. These were averaged to 
secure a final rating. 

The educational tests given were the three reading tests and 
the two arithmetic tests of the Stanford Achievement Advanced 
Examination, Form A. The total arithmetic score and the total 
reading score were used as a measure of their ability in these two 
subjects. 
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III. RESULTS 
a. Correlation of the non-verbal tests with each other 


The correlation between the raw score on the Pintner Non- 
Language Mental Test and the total weighted score on the 
Point Performance Scale was .80 +.01 (n = 333). The cor- 
relation between the score on the Pintner test and C.A. was .66 
+.02 (n = 337) and the correlation between the rating on the 
Point Performance Scale and C.A., was .74 +.01 (n = 390). 
With C.A. partialed out the correlation between the Point 
Performance rating and the non-language test was .61. This 
is a substantial relationship and indicates that the two tests 
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measure the same thing to a considerable degree. What they 
measure is another question. 

The high correlation with C.A. shows that the tests do differ- 
entiate one age level from another. The older the child the 
higher will be his score. The tests measure some kind of 
maturity. Whether this is a mental or physical maturity or a 
combination of the two cannot be determined from these 
results alone. 


b. The correlation of the non-verbal tests with school achievement 


Table 1 gives these correlations. It is interesting to note 
that the correlation of these tests with the other variables is 


TABLE 1 
Showing the correlation between the Pintner Non-Language Mental Test 
and the Point Performance Scores and other variables 
(n = 98) 





PINTNER POINT 
RAW SCORE PERFORMANCE 





Total Arithmetic Score on Stanford Achieve- 


Lia in kin ch es pa RC dp asics Chau 45 + .05 .37 + .06 
Average of Teachers marks................. .26 + .06 .21 + .06 
Total Reading Score.....................05. .003 + .07 | —.06 + .07 











about the same in each case. The correlations decrease with 
the amount of reading material entering into each. It is 
highest with arithmetic where a limited knowledge of English 
language is required and lowest with reading. With the ex- 
ception of arithmetic the tests do not predict to any appreciable 
degree school achievement. 


c. Correlation of non-verbal tests with estimates of general 

intelligence § 

The correlation between the raw score of the Pintner Non- 
Language Mental Test and the average of the ratings given by 
the teachers is .12 +.07 (n = 98) and the correlation between 
the total weighted score on the Performance tests and teachers’ 
judgment is .18 +.07 (n = 98). These are negligible cor- 
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relations. They are much lower than the correlation of teach- 
ers’ estimates of intelligence with the scores on verbal intelli- 
gence tests. In the latter case the correlation is usually around 
.50. It is evident from these results that teachers do not base 
their judgments of intelligence on the type of performance 
required in the non-verbal tests. If teachers’ judgments of 
intelligence can be taken as a fair sample of judgments in 
general, then these non-verbal tests do not measure that type 
of response that is usually considered as general intelligence. 
The fact that teachers’ judgments of ability have a closer 
correspondence with the verbal than with the non-verbal tests 
indicates that what is usually considered as general intelligence 
is the type of response that is associated with the use of language 
concepts. 


IV. CONCLUSIONS 


a. There is a close correspondence between the non-verbal 
tests. They measure the same thing to a fairly close degree. 
b. The test scores indicate the degree of some kind of ma- 
turity. That maturity may be physical or mental or a com- 


bination of these. 

c. With the exception of arithmetic these tests do not predict 
educational achievement. 

d. There is practically no relation between what is usually 
considered by teachers as general intelligence and the type of 
response measured by these tests. 
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The problem of growth has attracted the attention of people 
who are interested in scientific education for a number of years. 
It seems that if we knew and understood the growing child from 
a developmental point of view that the science of education 
might be extended to a much more desirable end. There is a 
great desire to study the biological and psychological develop- 
ment of the child along together rather than to attempt to draw 
a line of cleavage between the two. The writer has attempted to 
make a study of the physical growth of dull children to find out 
whether the dull Italian child grows as much from age to age as 
the normal Italian child. 

We will not attempt to review all of the literature bearing 
upon the relation between mental and physical development, 
but only some of the outstanding studies which might throw 
some light on the problem under consideration. One of the 
early studies was made by Porter! who selected a large number 
of children in the schools of St. Louis and studied the height, 
weight, length and breadth of head, vital capacity, and acute- 
ness of vision. He found a fairly constant ratio between weight 
and the grade in school for the ten-year-olds in the fourth grade 
were heavier than the ten-year-olds in the third. He concluded 
that “‘precocious children are heavier and dull children are 
lighter than mediocre children of the same age.”’ 

Christopher? published a study on school standing and height, 


1 Porter, W. T. The Physical Basis of Precocity and Dullness. 
Trans. Academy of Science of St. Louis, 1893, pp. 161-181. 

? Christopher, W. S. Measurement of Chicago Children. Journal 
of American Association, 1900, pp. 618-23, 683-87. 
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weight, strength of right and left grip, sitting height and vital 
capacity. He studied about 1000 children in Chicago ranging 
in age from four to sixteen. His findings are well stated in the 
following words, “at the outset of my investigation I determined 
to reinvestigate Porter’s proposition, and have to say that such 
facts as we have been able to collect go to confirm it.”” Smedley 
continued the study of Christopher and his results indicate the 
same conclusions, but added that dull pupils had a greater 
number of defects than the brighter children. Hastings made a 
study of the public school children in Omaha, Nebraska, which 
showed that children of the same sex develop mentally in a 
similar way to their physical growth such as height, weight, etc. 

Another investigation was made by Lapage who found that 
feeble-minded children were smaller than average children and 
were less developed physically. Meade studied about 288 boys 
and 141 girls in a feeble-minded institution in Indiana and 
found that mental defects affected the height and weight of 
children, the more decided the defect the more checked the 
physical growth. He also found that the moron boys were 
below the normal in height, but were taller than imbeciles and 
idiots. Similar conditions were found in the study ofthe girls, 
but the difference was not as great. Doll made a similar study 
and the results showed that the feeble-minded children were 
below the normal in sitting height, weight, and right and left 
grip. 

Baldwin studied the educational, mental and physical 
measurements of 129 pupils who were relatively inferior physi- 
cally. He found that there was no correlation between the 
educational and mental status, but a positive correlation 
between height, weight, and area of carpal bones and mental 
development. His study also showed that the mean mental age 
of the physiologically accelerated children was higher than the 
average mental age of the physiologically retarded children. 
Baldwin in a later study emphasized the importance of a careful 
study of mental and physical growth and pointed out the 
fact that many of the early investigators neglected the study 
of growth. ‘The day for the study of physical growth processes 
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by single measurements is over. Let us begin to do what 
should have been done long ago, plan consistently to make in- 
tensive consecutive studies throughout a series of years on the 
same individuals.’’ 

Juddlesohn studied the records of about 1524 mentally 
retarded boys and 538 girls taken from special classes in the 
schools. He found that the boys were shorter and lighter than 
normal boys for all ages, and that mentally retarded girls were 
shorter for all ages, and lighter from the ages of nine to twelve, 
and about the same weight for the ages of thirteen and fifteen 
years. 

The writer has made a study of the mental and physical 
growth of dull North European boys and girls from the ages of 
six to twelve in standing, sitting, sternal height, weight, leg 
length, iliac measure, and trunk length. The measures for 
these children have been taken annually in all of the above 
measures and the following conclusions have been drawn from 
the investigation. ‘“This study shows that group tests are not 
only objective measures, but are fairly reliable in selecting 
children for a scientific study of growth. Dull children are 
below the normal in all measures of physical growth from age 
six through eleven, except at age six in trunk length, and in iliac 
measure for the girls.’’ 

It seems that the majority of previous studies indicate that 
there is a relationship between the mental and physical status 
of children. There is a fair agreement that children who are 
below the normal in intelligence are also below the normal in 
physical growth. Most all of the children who have been 
studied are either North Europeans or mixed races. Racial 
differences have not been stressed. In this investigation we 
will consider only the Italian race as found in the Massachusetts 
public schools which should give us a fair picture of what we may 
expect of Italian children in the public schools of the country. 


* Baldwin, B. T. The Relation Between Mental and Physical 
Growth. Jour. Educat. Psychol., 1922, p. 193. 

* Wheeler, Lester R. A Comparative Study of the Growth of Dull 
Children. Jour. Educat. Research, October, 1929. 
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This study differs from most of the previous investigations in 
two ways. First the cases of previous studies have been selected 
on the basis of teachers’ marks or estimates, and second the 
physical growth measures have not been taken on the same 
individuals over a period of years. These two main differences 
are of vital importance in a scientific study of growth, and have 
been considered in this study in both the selection and the 
measurements in the different physical traits. 





























TABLE 1 
| BOoYs GIRLS | TOTAL 
| 
nN ERT ORE OEE SEE TENE: 122 125 247 
a ee ee 6-12 6-12 6-12 
PR Yin os oracdahaecsesheug eves 60-90 56-90 56-90 
I re a ots cuace occ ct be Dera 82.5 82.3 82.3 
| MES ETS ey SOL Peay ers A OE 85.8 86.1 85.9 
ihackr Hs 04 ino cs Kedk> Seen 77.5 77.8 77.7 
Sap epee ee ae oe 4.2 4.2 4.1 
TABLE 2 
1Q BOYS GIRLS TOTAL 
85-90 37 36 73 
80-85 41 39 80 
75-80 27 31 58 
70-75 S S 16 
65-70 S 4 12 
60-65 0 1 1 
55-60 1 1 2 
50-55 0 0 0 











The data for this study were obtained from the Harvard 
Growth Study under the direction of Dr. W. F. Dearborn’ to 
whom the writer is greatly indebted. These data have been 
taken from children in the public schools. A full and compre- 

5 Dearborn, Walter F. The Study of Individual Abilities—The 


Normal Child. School and Society, Vol. XVIII, Nos. 4, 5 and 6, Sep- 
tember 22, 1923. 
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hensive description of the different measurements and the 
technique used in securing them is given by Dearborn. The 
mental and physical measurements of the same children were 
made at consecutive periods of approximately twelve months 
intervals by individuals who were especially trained for this 
type of research. This is of vital importance because both 
mental and physical measurements should represent asnearly 
as possible the same stage of growth on the same individual for 
each year. Latshaw has also given a complete description of 
the different phases of the problem and the significance of the 
investigation as a whole. We are safe in saying that the 
children in this study will present a fair picture of what you will 
find in the public schools where there is an Italian population. 

The children included in this study were selected whose 
averaged IQ was below 90 as measured by the Dearborn Intel- 
ligence Tests for six consecutive years ranging from the ages 
of six to twelve. The children in this’study are principally dull 
since about 6 per cent fall below IQ of 70 as shown in table 1. 

Table 2 shows the distribution of IQ of Italian children. 

It is evident that the large majority of the cases used in this 
study are dull children or fall between 70 and 90 in IQ as shown 
in table 2. We would also like to point out the fact that the 
median IQ is only about eight points below the line of demarca- 
tion for normal children. This is a small difference in consider- 
ing a study of the physical growth of dull children. 

The standards for normal Italian children were established by 
Latshaw who took a number of cases according to age, race, and 
sex from data obtained from the Harvard Growth Study. The 
norms are probably lower than we might expect due to the fact 
that there was not a normal percentage of superior Italian 
children. It seems that according to our intelligence tests a 
larger percentage of the Italian children taken at random in the 
public schools fall below the normal. This might affect the 
standards for physical measures. 

There are many problems which might grow out of this study 
which we will not be able to answer. For example we do not 
know whether the differences are caused by heredity or environ- 
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ment, nor are wé able to answer the question of how much the 
children have grown between the annual measurements, nor are 
we able to say that this will hold true for the individual, but 
we do maintain that these findings should give us a general trend 
of the group. We will not be able to say whether the Italian 
child who is below the normal in intelligence will reach the 
adolescent period at the same time as the normal child or vice 
versa, but we will have to wait until measurements can be made 
on these children through the adolescent period, because our 
data only extends through the eleventh or to the twelfth year. 

The growth of the dull Italian boys as measured by a com- 
parison of the medians from year to year seems to be below the 
normal in most of the physical traits with the exception of the 
iliac measure. In the iliac measure there is quite a variation 
which might be due to the difficulty of exact measurement 
through the hips. The differences are small in all of the meas- 
ures and clearly show that the Italian boys could not be 
grouped on the basis of their physical growth as some investi- 
gators in physical growth have advocated. Figure 1 shows that 
the differences are not great but seem to be fairly consistent 
from age to age. There is no indication of growth spurts, but 
the growth curves seem to follow the gradual theory from age 
six through eleven or as far as our measurements have been 
taken. There is no indication of an acceleration of physical 
growth which might indicate an approach to the adolescent 
period. These data indicate that the dull Italian boys seem 
to be not only below the normal in intelligence but also retarded 
in physical growth in all measures except at the age of six, in 
height and trunk length and no significant difference in iliac 
measure. 

The medians of dull Italian girls seem to fall below the normal 
from year to year in most of the measures as shown in figure 2. 
In some of the measures at age six and eleven there seems to be 
no difference. Due to the small number of cases at ages six and 
eleven we can make no prediction for these ages. The differ- 
ences in physical growth between the normal and the dull girls 
are small and would not be great enough to group the children 
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for special instruction in school but those children who deviate 
widely from the normal should have special consideration in 
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grouping for physical and mental training. Figure 2 shows 
that the dull Italian girls are below the normal medians in most 
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of the measures in standing, sternal, sitting height, weight, 
trunk and leg length, with no material difference in iliac 
measure. 
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Fig. 2. CoMPpARISON OF THE MEDIANS OF NORMAL AND DULL ITALIAN 
GIRLs IN PuysicaL GROWTH 


Table 3 shows the semi-interquartile range of the normal and 
dull Italian children from age six through eleven which gives us a 











LESTER R. WHEELER 


384 















































































































































pace er. Cet Cel eee St SCT S SS Oey Stl tei Sei Ft ae AAs id 
692; 2h) 2E)] FS) GS) SI; Lt} 9s) FS) 8s) 9S) TU] SI] 9c yeUION 
Vq3IOM 
98°I Z g'T 9°I 81 SI 81 | O8'T g'I 0% 9°I eI eI ae LL ma 
wri 8S) FEi Stl 2Ui Cli Sere tt Ost Cre etry Ort etre’ [SULION 
VysIey Buyy1g ~ 
ore | Sh) TE) Fs; 22) 9S) LE) ke) SS] FE; SS) fe) GS) Sey "Ted 
me | SH) LE; Fe] $s; oF] OF)/ SOE] SE) 9E| SS) Fe] 6S; FE)" use alle [SULION 
WSrey [BUIIg 
sce; Tg; OF] TE] Fel] 9S!) Ze eee| se; 98) 6S) Fe) Se) wey” yo “Ted 
868) TS) oF) GE! LE] 9E| FE S9E| FH) GE; 9S) OF) FE; Ber’ **"TSGLION 
ysteqy Surpusys 
9 POL | 86 86 £8 oY & 69 TOL | 26 ol 65 TIMP S989 Jo Joquin Ny 
Or | 68ST | &% | LIT | 822 | FST LOT | 11% | OS@ | HEL | 18 | OI | *"""TSaniog 
Sesvo jo JequInNN 
“A’'V |s1e04 []|81804 01) S1v0A 6 | SIBVOA g | GIVOA 2 | BIVOAQ| “AY |[8IBOA [| \8I180A C1) BIVOA G | SIVOA Q | BIVOA J | BIBOA GQ 
sTaID $1048 











uaip}ryo uDY~D]T POUWLOU pun pnp fo buns a2ysvnbsaqur-rmagy 


€ TIAVL 





385 





£3" , : : ‘ , aouslegIg 
ths ! A’V TIP [890.1 
19% j ’ “AY [eUlio” [830], 





8°0 
60 















































GIPrs Ovi] 





TST) PI] v 
¢" 


I 
TS't| ZT T 











qy3Zue] yunsy, 





Z 
2] 
ion 
a 
=) 
_ 
>] 
oO 
Z 
- 
— 
- 
«A 
> 
_ 
=| 
<) 
b 
a 
Mm 
° 
F 
° 
io) 
1) 
= 
< 
oO 
_ 
2) 
~ 
ee] 
mo 


Teo) €6)| 3 
ws) 9S) 8S 























yy3ue] Ze'T 











386 LESTER R. WHEELER 


fair picture of the spread of the middle fifty per cent of the 
normal and dull groups. An examination of the semi-inter- 
quartile range shows that the dull group follows fairly close to 
the normal in the different measurements for all of the ages. 
There is a narrower range found in the dull group which is due 
probably to the small number of cases. The quartile deviation 
is fairly constant and strengthens the reliability of the mediars 
as shown in figuresland2. Weare fairly safe in saying that the 
dull group with a much smaller number of cases for the different 
ages studied approaches a normal distribution in the different 
physical measures taken and therefore indicates a small but 
fairly consistent difference in the medians for the two groups. 


THE RELIABILITY OF THE DIFFERENCES 


We have compared the dull and normal groups in terms of the 
medians and the quartile deviations. We will next discuss the 
reliability of the differences of the groups. If we increase the 
number of cases of dull Italian children what may we expect in 
terms of the reliability of the differences between the medians? 
The method of obtaining the probable error of the differences 
was taken from Garrett® and the P.E. was calculated for the ages 
of seven, nine, and eleven which should give us a fair sampling 
of the dull children. Table 4 gives the differences between the 
medians, the P.E. of the difference, the difference divided by 
the P.E. and the chances in one hundred if we increase the 
number of cases. 

The P.E. of the difference shows in table 4 a small and fairly 
consistent difference above mere chance for all ages in the differ- 
ent physical measures except age eleven in standing height, and 
nine and eleven in iliac width. The chances are 75 in 100 that 
these measures will hold true for a larger number of cases as 
shown in the average for the entire group for all ages and meas- 
ures. The reliability of the measures of the Italian boys in- 
dicate a trend of a small consistent difference in most of the 
general measures as shown in figure 1. 


*Garrett,H.G. Statistics in Psychology and Education, pp. 135-137. 
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TABLE 5 


Percentage of dull Italians below normal medians 





wa | 


BOYS 


GIRLS 





Standing height 






































Below Below 

6 50 50 

7 58 60 

8 59 60 

9 58 50 

10 74 72 
11 58 53 
A.V. 60 58 

Sternal height 

6 55 52 

7 63 65 

8 63 63 

9 60 56 

10 72 74 
11 56 57 
A.V. 62 61 

Sitting height 

6 56 50 

7 57 60 

8 60 58 

i) 60 56 

10 62 80 
11 64 50 
A.V. 57 59 

Weight 








52 
59 
54 
62 
77 
61 
61 
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TABLE 5—Concluded 
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Table 4 shows a difference above mere chance in the majority 
of the measures taken on the dull Italian girls for all ages 
studied. The chances are 75 in 100 that these results will hold 
true for a greater number of cases as shown in table 4. 

Overlapping is measured by the per cent of the dull group in 
the different physical traits below the medians of the normal 
children. The percentage of overlapping gives another check 
on the median differences of the two groups and shows the 
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trend of the dull Italian children. The method of obtaining the 
per cent of overlapping is found in Garrett.’ Table 5 shows 
that there is no material difference in the per cent of overlapping 
in the seven different physical traits measured in the Italian 
boys as shown by the average of each trait. The differences 
between the dull and normal groups are fairly constant for all 
ages and give an average per cent below the medians of the 
normal group for all traits of about 58.2 per cent, in other words 
41.8 per cent of the dull Italian boys are above the median of 
normal Italian boys in the different physical measures studied. 
Table 5 shows that the Italian dull girls follow a general trend 
similar to boys, but a smaller per cent below the normal medians 
for the different physical traits from age six through eleven. 
The total average per cent of all traits and for all ages is 57.5 per 
cent, which is a small difference but consistent, which indicates 
that there is a fairly constant difference between dull and normal 
Italian children in physical growth. 

This study shows that there is not as great a difference in 
physical growth between normal and dull Italian children as 
there is among North European children as shown in a previous 
article.* The difference in growth is not as marked as many 
writers have advocated, but does show a small and fairly 
consistent difference. From the writer’s study of the problem 
of growth between dull and normal children the difference is not 
great enough to classify on the basis of physical age, but it 
should be considered as a factor in the classification, instruction 
and in promotion especially when the child falls in either 
extreme of the distribution or deviates widely from the normal 
in physical growth. 


SUMMARY AND CONCLUSION 


1. The majority of previous studies indicate that there is a 
definite relationship between mental and physical development. 
2. Most of the previous studies have been based on children 
7Garrett,H.G. Statistics in Psychology and Education, pp. 44-49. 


8 Wheeler, Lester R. Comparative Study of the Physical Growth of 
Dull Children. Jour. Educat. Research, Oct., 1929. 
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selected at large and no attention has been given to the study of 
dull Italian children selected on the basis of group intelligence 
tests. 

3. This study shows that there is a small and fairly consistent 
difference in the majority of measures studied between dull and 
normal Italian children in standing, sternal and sitting height, 
weight, trunk length, and iliac width from age six through 
eleven. 

4. That the difference is not great enough for classification 
on the basis of physical growth, but should be considered as a 
factor in the classification, instruction, and promotion of chil- 
dren who deviate widely from the normal in physical growth. 
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NEWS AND NOTES 


A CASE OF PLAGIARISM? 


In the March 1926 number of the JournNat or AppLizep PsycHo.oery,' 
Dr. Walter Scott McNutt published an experimental article in which he 
used, in the section devoted to the discussion of results, phrases and sen- 
tences strikingly similar to those appearing in Titchener’s Textbook of 
Psychology. Dr. McNutt stated, in a subsequent note in which he 
attempted to explain the similarity,? that he did not know of Titchener’s 
discussion. Those who know Dr. McNutt best claim, moreover, that he 
‘is not the kind of man who could be guilty of plagiarism,’’ and they 
vouch for the fact that he “‘did not know that he was using Titchener’s 
language.” 

In view of these statements the case becomes of peculiar interest to 
psychologists. Is the similarity an accident? Does the case illustrate 
chance association? May it be explained by the community of ideas 
and the convergence of associations? In order that the reader may 
judge for himself, the parts of Dr. McNutt’s paper that show the re- 
markable similarity are set opposite the sections in Titchener. 


McNort’s ARTICLE TITCHENER’S TEXTBOOK 
Page 127 Page 364 
3rd para. 2nd para. 
The simplest kind of perception, The simplest kind of perception, 


pure perception implies the group- then,—what we may call the pure 
ing of sensations under the lawsof perception,—implies the grouping 
attention. But they seem to be of sensations under the laws of at- 
made up of more than is perceived tention. But it is clear that per- 
to the eye, skinandear. Thesen- ceptionsare, asa rule, not made up 
sations gotten from the stimuliare solely of sensations; we see and 
supplemented by images, and the hear and feel more than is pre- 
perceptions become mixed percep- sented to eye and ear and skin; 
tions, including complexes of sen- the given sensations are supple- 
sory and imaginal elements. mented by images. Most of our 
perceptions are mixed perceptions, 
complexes of sensory and imaginal 
elements; and the life of percep- 
tion is, for more than one is apt to 
suppose, a life of imagination. 





1 Op. ecit., 10, 1926, 117-128. 
2 Ibid., 13, 1929, 316. 
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Page 127 
4th para. 


But that is not all; the percep- 
tion had meaning. No sensation 
means; a sensation simply goes on 
in various attributive ways, spa- 
tially, clearly, intensively, etc. 
All perceptions mean; they go on 
also in various attributive ways, 
but they go on meaningly. 


Page 127 
5th para. 


From a psychological stand- 
point meaning is always context; 
one mental process is the meaning 
of another mental process if it is 
that other’s context. And con- 
text, in the sense used here, is 
simply the mental process which 
comes as a natural result to the 
given process through the situa- 
tion in which the subject finds 
himself. At first, the situation is 
physical and external, and origi- 
nally the meaning is kinaesthesis. 
The subject faces the situation by 
a bodily attitude; and the charac- 
teristic sensations which the atti- 
tude arouses give meaning to the 
process which stands to thecon- 
scious focus; and they are there- 
fore psychologically the meaning 
of the process. The situation may 
be either external or internal, 
either physical or mental, either a 
group of adequate stimuli or a 
constellation of ideas. Image has 
now supervened upon sensation, 
and meaning can be carried in- 
to imaginal terms. Therefore, 


Page 367 
lst para. 


But that is not all; the essential 
thing about them has still to be 
named: and it is this,—that per- 
ceptions have meaning. No sen- 
sation means; a sensation simply 
goes on in various attributive 
ways, intensively, clearly, spa- 
tially, andsoforth. All perceptions 
mean; they go on, also, in various 
attributive ways; but they go on 
meaningly. 


Page 367 
2nd para. 
Meaning psychologically, is al- 
ways context; one mental process 
is the meaning of another mental 
process if it is the other’s context. 
And context, in this sense, is sim- 
ply the mental process which 
accrues to the given process 
through the situation in which the 
organism finds itself. Originally, 
the situation is physical, external; 
and originally, meaning is kin- 
aesthesis; the organism faces the 
situation by some bodily attitude, 
and the characteristic sensations 
which the attitude arouses give 
meaning to the process which 
stands at the conscious focus, are 
psychologically the meaning of 
that process. For ourselves, the 
situation may be either external or 
internal, either physical or mental, 
either a group of adequate stimuli 
or a constellation of ideas; image 
has now supervened upon sensa- 
tion, and meaning can be carried 
in imaginal terms. For us, there- 
fore, meaning may be mainly a 
matter of sensations of the special 
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meaning for us may be mainly a 
matter of sensations of the special 
senses, or of images, or of kinaes- 
thetic or other organic sensations, 
as the situation under which we 
are placed demands. 


Page 128 
2nd para. 


But in order to have meaning 
there must be two sensations. 
When the sensation of it is accom- 
panied by a strain, it becomes a 
perception of color in the place of 
a mere reaction to stimulus. It 
becomes that gray something; and 
it owes the that something to the 
strain content. 


Page 128 

2nd para. 
However, meaning is not always 
conscious; it may be carried in 
purely psychological terms. 


Page 128 
2nd para. 


A word, facial expression and in- 
flection of the voice, a bodily atti- 
tude, a stimulus of any kind may 
press the nervous button, and con- 
sciousness is turned, unknowingly, 
into new channels. 
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senses, or of images, or of kinaes- 
thetic or other organic sensations, 
as the nature of the situation 
demands. 


Page 368 
4th para. 

The gist of this account is that 
it takes at least two sensations to 
make a meaning. If the sensation 
of light is accompanied by a strain, 
it becomes forthwith a perception 
of light, with meaning; it is now 
‘that bright something; and it 
owes the ‘that something’ to its 
strain-context. 


Page 369 

3rd para. 
But is meaning always conscious 
meaning? Surely not: meaning 
may be carried in purely psycho- 
logical terms. 


Page 369 
3rd para. 


The course and connection of ideas 
may be determined beforehand 
and from without; a bodily atti- 
tude presses the nervous button, - 
and consciousness is switched, 
automatically, into new channels. 


If the similarity in the choice of words and sentence structure is due to 
chance the case is one of the most remarkable instances of chance asso- 


ciation on record. 


Martin L. Rermert, 
Wittenberg College. 





NEWS AND NOTES 395 


Editor’s Note: The editor of this Journat has printed the above 
introductory and parallel statements against his better judgment. To 
permit the above statement to appear in another journal would make it 
appear that this JourNAL is unwilling to do all that it can to discourage 
the all too common practice of plagiarism and closely allied practices. 
We print below a few excerpts from our correspondence concerning this 
unfortunate affair. 

From Dr. MeNutt’s early correspondence we learn that he thought 
that if his bibliography had been published in addition to certain itali- 
cised words the sources of his material would have been clear to the 
reader. He also stated that the manuscript had first been submitted to 
Dr. Titchener for publication in the American Journal of Psychology but 
had been returned to him with the statement that they had already too 
much material on hand. The suggestion was made that he submit it to 
other Journals including our own. It appears later that Dr. Titchener 
turned this manuscript over to Dr. Dallenbach who returned it to the 
author. 

In a letter addressed to Dr. Reymert Dr. Titchener stated that while 
he considered plagiarism a serious matter a ‘paragraph of apology’”’ from 
Dr. McNutt would be a satisfactory solution. Following a trip to 
Europe, and therefore a regrettable delay, we secured a paragraph of 
apology (This Journat, June, 1929). This was submitted to Dr. Reymert 
with the suggestion that he might wish to refer it to some member of Dr. 
Titchener’sfamily. Dr. Reymert assured us that he had passed upon the 
matter himself, and that the name of the professor from whose lecture 
the material was obtained be printed and that certain statements be 
deleted. The sentence crossed out by Dr. Reymert was not printed and 
we deleted also a reference by McNutt to an unpublished manuscript. 
In the final approval of what was to be printed there came the first men- 
tion of the name of Dr. Perrin. We had mistakenly inferred that Dr. 
Tawney was the lecturer whose notes had been wrongly used. We im- 
mediately wrote Dr. Perrin but received no reply. It turns out that he 
was traveling and the letter did not reach him. Inasmuch as Dr. Rey- 
mert had suggested that the name of the professor concerned be printed 
we proceeded with the printing of the paragraph of apology. 

From further correspondence with Dr. McNutt made advisable because 
of the insistence by Dr. Reymert that the above parallel statement be 
printed, we learned that Mr. McNutt not knowing of Dr. Titchener’s 
death wrote to him directly, making further apology and asking if he 
would not consider publishing the second manuscript on another phase 
of the same subject earlier submitted to us. 

Upon the receipt of a letter from Dr. Perrin together with a copy of his 
statement sent for publication in this Journat (June issue, 1930), Dr. 
Reymert wrote to Dr. Perrin that he was “‘telling’’ the editor of this 
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Journal not to print Perrin’s statement inasmuch as he desired to incor- 
porate it in his own account. Such action as this the editor resented. 
and immediately wrote to Dr. Reymert to that effect. 

After an unsuccessful attempt to arrange a conference including Drs. 
Reymert, Perrin, Tawney and the editor of this JouRNAL an agreement 
was reached between Dr. Reymert and the editor that the publication 
of Dr. Perrin’s statement would meet the situation. Following this 
agreement, however, it was learned that the above brief statement and 
parallel comparison would appear in another psychological journal. 

Our last word from Dr. Reymert was an order for 200 reprints of the 
statement printed above. He was then informed that in addition to his 
statement the notes herewith appended would also be printed. No 
further word has been received from him. 


Dr. Truman Lee Kelley, Professor of Education and Psychology at 
Stanford since 1926, has accepted an appointment as Professor of Edu- 
cation in the Graduate School of Education, Harvard University. Dr. 
Kelley will offer at Harvard courses in Advanced Statistical Method and 
a Seminar in Psychometrics. His special research interest lies in the 
analysis of mental structure and functions. By the experimental use 
of tests and statistical procedure he is seeking to isolate and study the 
basic traits, powers, or abilities that enter as unitary components into 
the complex processes of learning and conduct. This is obviously a 
fundamental problem for education, guidance and social control. 


Dr. Donald G. Paterson, of the Department of Psychology at the Uni- 
versity of Minnesota, contributes an essay on Personality and Physique 
to the symposium by four Minnesota professors which was selected by 
the Scientific Book Club as its “book of the month’’ for August. The 
volume is ‘‘The Measurement of Man,’’ studies in biometrics which will 
be published by the University of Minnesota Press in August. The 
other authors are the late Professor J. Arthur Harris and Professors 
Clarence M. Jackson and Richard E. Secammon. 


Hulsey Cason, University of Rochester, has accepted an appointment 
as Professor of Psychology at the University of Wisconsin for the next 
academic year. 


Stanford University makes the following announcements: Dr. W. R. 
Miles will spend his sabbatical year 1930-31 at the Institute of Human 
Relations, Yale University. Dr. L. P. Herrington has a Social Science 
Fellowship for 1930-31 for study at the University of Marburg. Mr. 
Floyd Ruch has a National Research Fellowship for 1930-31 for study at 
the Sorbonne. Mr. E. Lowell Kelly becomes Assistant Professor of Edu- 


ce 
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cation and Psychology at the University of Hawaii while Mr. Harry 
Harlow becomes Assistant Professor of Psychology at the University of 
Wisconsin. 


Victor H. Noll, instructor in Educational Psychology at the Univer- 
sity of Minnesota, has been appointed Educational Specialist with the 
National Survey of Secondary Education, under the auspices of the 
Office of Education of the United States Department of the Interior. 
He will begin his new work August Ist. 


Ohio University announces the appointment to the Department of 
Psychology for the coming academic year of Dr. C. M. Louttit, of Yale, 
who for the past two years has been connected with the University of 
Hawaii. Dr. Gaige B. Paulsen, of the University of Minnesota, has 
also accepted an appointment to the staff of the Department of Psychol- 
ogy. Professors Amos C. Anderson and Starke R. Hathaway have been 
granted leaves of absence for the coming year. 











BOOK REVIEWS 


Benno Kern. Wirkungsformen der Ubung. Helios, Minster, 1930. 
ix + 508 pp. 

The monograph recounts experiments on the effect of practice with 
several simple tasks. The first involved a horizontal row of dots with 
intervening vertical lines extending somewhat above them. The sub- 
ject connected the dots with a pencil but had to trace up over the tops 
of the vertical lines. A variation involved similar dots but with a more 
complicated labyrinth around them. A second task consisted of simple 
addition with columns of five-digit numbers. A third comprise the 
first 25 letters of the alphabet in an array of 5 rows and 5 columns but 
in a random order. The subject located the letters in proper sequence 
and wrote after each the appropriate number,—‘‘Al, B2, C3’’ ete. The 
last task was cancelling a prescribed letter in a page containing four 
columns of capitals. 

In collaboration with several other investigators practice series were 
run for several weeks with subjects of a considerable age range and with 
variations of technique. The data are presented in considerable detail 
and discussed at length. Several ‘laws’ are formulated as a result of 
this analysis. For instance there is the “ontogenetic law of practice’’ 
that the younger the subject the smaller his initial score and the greater 
his improvement on a percentage basis. This holds within the age range 
from 9 to 20. Moreover within the same age group larger initial scores 
are associated with smaller improvement but this holds only for the 
very first scores and not the average for the first week orso. The author 
concludes that there is a sort of initial inhibition (Anfangshemmung) 
which makes a difference between the very first and the other early 
trials. He does not find in his experiments a tendency toward ‘‘leveling’’ 
or greater homogeneity of subjects’ performances after practice. After 
the first week the variability of the group changes little. Distributed 
repetitions in general prove superior. 

The author considers his most important contribution to be the fact 
that the relative efficiency of individuals after considerable practice is 
markedly different from their relative efficiency in the initial trial. He 
believes this is due to the initial inhibition mentioned above which con- 
stitutes an additional variable at the outset. He also speaks of ‘‘psychic 
friction’ that operates differentially in the early stages of practice. 
These findings have a certain bearing on the mental test program. In 
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most testing projects we give one rather brief trial. It would appear 
that this initial trial is not typical of what we would get after extended 
practice. The author thinks that about all we can hope to do with tests 
administered as at present is to divide subjects into a good and a poor 
group and the former will actually contain a larger percent of good in- 
dividuals. Individual prognosis is out of the question. 

Psychologists in the testing field are of course aware of these limita- 
tions. But it may not always be necessary, as the author implies, to 
practice each test several weeks before using the scores. The ‘‘initial 
inhibition’’ he talks about is often taken care of by a ‘‘shock absorber.”’ 
And while subjects’ initial and final scores in a test may differ, if a fairly 
brief trial has high reliability and validity in the statistical sense it may 
be used for vocational or academic prognosis. In some tests it has 
been found that correlations between test and criterion were higher 
after practice. Hollingworth noted this long ago. But this may not be 
universally true and as an empirical fact many projects have resulted 
in fairly accurate prognoses on the basis of tests. Furthermore there 
are many tests in which extended practice would be tantamount to 
memorizing the individual items,—an opposites test for instance,— 
unless unlimited material were prepared. Most of us do run a practice 
series with a new test that lends itself to repetition and we nearly always 
check a test for its statistical reliability. It seems to the reviewer that 
the author is needlessly alarmed. 

The experimental discussion is preceded by an extensive theoretical 
introduction setting forth the problem. The bibliography of /428 titles 
may prove interesting to workers in this field who are not very familiar 
with the German literature. The book is a thorough discussion of a 
rather simple series of experiments and its main contribution seems to 
be a warning to those using mental tests to take some account of the 
practice effect. 

Harotp E. Burtt, 
Ohio State University. 


Autrrep Apter. The Pattern of Life published by the Cosmopolitan 
Book Corporation, New York, 1930. 

This book with an introduction by W. Beran Wolfe, the editor of 
Adler’s clinical cases, demonstrates the method and technique of ‘‘In- 
dividual Psychology’’ so characterized by the Adler school of neurosis 
and psychotherapy. It does this by introducing the reader to the 
clinical procedure used by Adler in handling cases of wayward and de- 
fective children. The whole book is filled with good common sense plus 
keen scientific insight into the problems of maladjustment. 

The cases presented to Dr. Adler while conducting his series of 
clinic-lectures at the New School of Social Research in New York during 
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the year 1929, were first the records of the mal-adjusted cases as brought 
to him by New York physicians, psychologists and teachers. These 
records were prepared by the persons presenting the cases according to 
the protocol used by Adler in his own clinic in Vienna. The case 
histories were reviewed and analyzed by Adler before his students in 
the lecture room; then the parents of the child were brought in and ques- 
tioned and instructed before the class; after which the child was brought 
in and the situation discussed with him in simple kindly language. 
The follow-up work, as indicated by the analysis, was then entrusted 
to the teacher or physician who had presented the case. Such cases as 
“The Boy Who Wants To Lead,”’ ‘“The Fear Of Growing Up,” “‘Con- 
genital Feeble-Mindedness,”’ etc., are treated and the analysis of these 
cases are recorded in the book in a non-technical language which a lay- 
man may read with profit. 

The Pattern of Life is filled with the dynamics of living children who 
are trying to make life adjustments and who, when failing, use all instru- 
ments and substitute devices they have or can muster in order to gain 
the recognition, the appreciation, and the understanding necessary for 
their happiness from their parents, from their brothers and sisters and 
from the rest of their little societal world. Barring some very loose 
usage in terminology such as, ‘‘patterns,” ‘‘goals of life,"’ ‘‘soul,’’ etc., 
all of which phenomena so-labelled could be explained it seems to the 
writer in more precise language such as “‘needs to be met,’’ “‘trial and 
error” behavior leading to habit formation, etc., it is a good book and 
written in an interesting style. 

James R. Patrick 
Ohio University. 


Harry Dexter Kitson. How to Find the Right Vocation. Harper and 
Brothers Publishers, New York, 1929. x + 202 pp. 

The need for a simple treatment of the facts concerning vocations 
and the selection of one can scarcely be denied. The difficulties of being 
both popular and scientific being what they are, the author has met the 
need remarkably well. He begins quite in the best tradition with a 
refutation of the claims of fortune tellers, following one Ben Underwood 
on a wearying journey from palmist to character analyst until having 
the good fortune to meet a student who is taking a course in psychology, 
he is set upon the right track. How to go about the exploring of various 
occupations, the value of a study of biography and its limitations, that 
worth while jobs require special training, how to get a job, these are some 
of the subjects discussed. On the whole the topics are well selected and 
the treatment simple, wholesome, and to the point. There are sore 
inconsistencies as when the author tells us on page 28 that most people 
in their selection of an occupation show a history of progressions, watch- 
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ing the changes in society and adjusting themselves accordingly, and on 
page 30 that most people fall into their jobs by accident. There is the 
occasional sweeping generalization as when he tells us what percentage 
of people are misplaced in their occupations or how many choose their 
occupation once for all in childhood. To be sure the writer protects 
himself with a cautious ‘“‘perhaps’’ but this does not make his figures the 
less deceiving if his guess happens to be wrong. Thereis the occasional 
attempt to be impressive, as when in speaking of his comparison of 
people who changed jobs often with those who stayed long in the same 
job, the author informs us that ‘‘On making elaborate calculations with 
the most refined statistical methods, I found that on the average these 
two groups were making the same amount of money.’’ While this sort 
of thing is a bit funny to the adult reader, it may duly impress the youth- 
ful mind and showmanship, after all, has its place in the world. A 
number of illustrations, done in the humorous vein, give the book spice 
perhaps for the juvenile reader but add little if anything to the value 
of the treatment. Indeed it may be suggested that they at times over- 
emphasize unimportant points and are for this reason misleading. 
None of the above criticisms would have any point had the author 
let the work stand frankly as the work for adolescents that it so clearly 
is. But when he says in his preface that ‘‘It will also meet the needs of 
professional counselors and teachers in educational institutions who 
give formal courses in ‘Choosing a Vocation’,’”’ he raises it to the rank 
of a text or reference book. Such a book should either present a con- 
siderable amount of new information or show some originality of organ- 
ization, and should be written with great respect for scientific accuracy. 
The book will serve admirably as a guide to those who know but little 
about occupations and who want to find their way into a suitable one. 
A. C. ANDERSON, 
Ohio University. 
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Learning Exercises. Laurence F. Suarrer, R. G. Simpson, Gien U. 
CrizeTon, Hazet Knigut and Max Scuogen. Harper and Brothers, 
New York. 38 pp. 

The Measurement of Manual Dezterities. F. M. Earnie and F. Gaw. 
National Institute «f Industrial Psychology, Report 4. Aldwych 
House, W. C. 2, London, England. 88 pp. 

Mind at the Crossways. C. Luoyp Morean. Henry Holt Company, 
New York. 275 pp. 

Ohio Conference of Statistics: Report of April, 1930. Bureau of Business 
Research, College of Commerce, Ohio State University, Columbus, 
Ohio, 50 cents. 67 pp. 

Our Knowledge of Other Minds. W. Wrure Spencer. Yale University 
Press, New Haven, Conn. $2.00. 145 pp. 

The Psychology of Childhood. Mary Scuarures. Richard R. Smith, 
New York City. 194 pp. 

The Psychology of Childhood. EpGcar James Swirt. D. Appleton and 
Company, New York. $2.25. 431 pp. 





* Mention here does not preclude further comment. 
402 





NEW BOOKS AND PAMPHLETS RECEIVED 403 
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Pavut V. Sancren. World Book Company, Yonkers, N. Y. 28 pp. 

Social Factors in Medical Progress. Bernuarp J. Stern. Columbia 
University Press, New York City. 135 pp. 

Social Research, Introduction to. Howarp W. Opum and KATHARINE 
Jocuer. Henry Holt & Company, New York. $4.00. 488 pp. 

A Survey and Program for Special Types of Education. Public Schools 
of Trenton, N. J. 96 pp. 
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Maine Studies, Second Series, No. 14. University of Maine, Orono, 
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